Sep 16 14:05:54.170958 ip-10-0-138-80 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 16 14:05:54.170975 ip-10-0-138-80 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 16 14:05:54.170985 ip-10-0-138-80 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 16 14:05:54.171310 ip-10-0-138-80 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 16 14:06:04.263866 ip-10-0-138-80 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 16 14:06:04.263886 ip-10-0-138-80 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot c5aa385e16e24a05be2e328a03d5ef1a --
Sep 16 14:07:57.107302 ip-10-0-138-80 systemd[1]: Starting Kubernetes Kubelet...
Sep 16 14:07:57.491983 ip-10-0-138-80 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:07:57.491983 ip-10-0-138-80 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 16 14:07:57.491983 ip-10-0-138-80 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:07:57.491983 ip-10-0-138-80 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 16 14:07:57.491983 ip-10-0-138-80 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 16 14:07:57.492635 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.492026    2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 16 14:07:57.496680 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496665    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.496680 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496680    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496684    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496687    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496690    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496692    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496695    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496698    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496701    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496703    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496706    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496709    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496711    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496714    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496717    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496719    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496723    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496726    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496728    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496731    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496733    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.496747 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496736    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496739    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496741    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496744    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496746    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496749    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496752    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496755    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496758    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496760    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496763    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496766    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496768    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496771    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496774    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496776    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496779    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496781    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496784    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.497233 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496787    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496791    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496795    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496799    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496803    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496806    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496809    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496812    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496815    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496817    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496820    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496822    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496825    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496828    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496830    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496834    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496837    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496840    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496842    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.497702 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496845    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496847    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496850    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496852    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496855    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496857    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496860    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496863    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496865    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496868    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496870    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496873    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496877    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496879    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496882    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496884    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496887    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496890    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496894    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496898    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.498171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496900    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496903    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496906    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496909    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496911    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496914    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496917    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496919    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496923    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496926    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496929    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496931    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496934    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496937    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496939    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496942    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496945    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496947    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496950    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.496953    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.498684 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497628    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497634    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497638    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497641    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497644    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497647    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497649    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497652    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497655    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497659    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497661    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497664    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497667    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497669    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497672    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497674    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497677    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497679    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497682    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497685    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.499222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497688    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497691    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497694    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497696    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497699    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497701    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497704    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497707    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497709    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497712    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497715    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497717    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497720    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497725    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497728    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497731    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497734    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497737    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497739    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.499716 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497742    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497745    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497747    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497750    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497753    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497756    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497758    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497761    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497764    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497766    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497769    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497773    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497777    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497782    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497785    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497788    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497791    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497794    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497797    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.500197 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497799    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497802    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497805    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497807    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497810    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497813    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497815    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497818    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497821    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497825    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497828    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497831    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497834    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497836    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497839    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497841    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497844    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497847    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497849    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.500666 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497852    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497854    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497857    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497859    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497862    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497864    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497867    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497870    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497873    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497876    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497878    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497881    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497883    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497886    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497888    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497891    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497893    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497896    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497899    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497901    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.501139 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497904    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.497906    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498295    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498301    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498304    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498307    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498310    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498313    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498315    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498318    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498322    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498324    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498327    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498330    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498333    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498336    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498339    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498341    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498344    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498347    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.501624 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498350    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498352    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498355    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498358    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498360    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498363    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498366    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498368    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498371    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498373    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498376    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498378    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498381    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498384    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498386    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498389    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498391    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498394    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498396    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.502127 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498400    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498404    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498407    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498410    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498414    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498418    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498421    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498424    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498427    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498430    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498433    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498436    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498439    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498442    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498445    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498447    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498450    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498452    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498455    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498457    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.502596 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498460    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498462    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498466    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498468    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498471    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498474    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498476    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498479    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498481    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498484    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498486    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498489    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498491    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498494    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498497    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498500    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498503    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498506    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498509    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.503091 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498512    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498515    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498517    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498520    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498523    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498525    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498528    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498531    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498533    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498536    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498539    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498541    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498544    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498546    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498549    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498553    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498557    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498560    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498563    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.503585 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498565    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498568    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498570    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.498573    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498647    2580 flags.go:64] FLAG: --address="0.0.0.0"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498679    2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498685    2580 flags.go:64] FLAG: --anonymous-auth="true"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498690    2580 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498694    2580 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498698    2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498702    2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498706    2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498711    2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498717    2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498720    2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498724    2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498727    2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498730    2580 flags.go:64] FLAG: --cgroup-root=""
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498733    2580 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498736    2580 flags.go:64] FLAG: --client-ca-file=""
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498739    2580 flags.go:64] FLAG: --cloud-provider="external"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498742    2580 flags.go:64] FLAG: --cluster-dns="[]"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498746    2580 flags.go:64] FLAG: --cluster-domain=""
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498749    2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 16 14:07:57.504093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498752    2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498756    2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498759    2580 flags.go:64] FLAG: --container-log-max-files="5"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498763    2580 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498766    2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498769    2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498772    2580 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498775    2580 flags.go:64] FLAG: --contention-profiling="false"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498778    2580 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498781    2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498784    2580 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498787    2580 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498791    2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498794    2580 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498796    2580 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498800    2580 flags.go:64] FLAG: --enable-load-reader="false"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498803    2580 flags.go:64] FLAG: --enable-server="true"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498806    2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498810    2580 flags.go:64] FLAG: --event-burst="100"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498813    2580 flags.go:64] FLAG: --event-qps="50"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498816    2580 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498821    2580 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498825    2580 flags.go:64] FLAG: --eviction-hard=""
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498828    2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 16 14:07:57.504677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498831    2580 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498834    2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498837    2580 flags.go:64] FLAG: --eviction-soft=""
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498840    2580 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498843    2580 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498846    2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498849    2580 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498852    2580 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498855    2580 flags.go:64] FLAG: --fail-swap-on="true"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498860    2580 flags.go:64] FLAG: --feature-gates=""
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498864    2580 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498867    2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498870    2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498873    2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498876    2580 flags.go:64] FLAG: --healthz-port="10248"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498879    2580 flags.go:64] FLAG: --help="false"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498882    2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498885    2580 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498888    2580 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498891    2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498894    2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498898    2580 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498901    2580 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 16 14:07:57.505263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498903    2580 flags.go:64] FLAG: --image-service-endpoint=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498906    2580 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498909    2580 flags.go:64] FLAG: --kube-api-burst="100"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498912    2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498915    2580 flags.go:64] FLAG: --kube-api-qps="50"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498918    2580 flags.go:64] FLAG: --kube-reserved=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498922    2580 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498925    2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498928    2580 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498931    2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498934    2580 flags.go:64] FLAG: --lock-file=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498937    2580 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498940    2580 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498943    2580 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498948    2580 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498951    2580 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498954    2580 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498957    2580 flags.go:64] FLAG: --logging-format="text"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498961    2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498965    2580 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498967    2580 flags.go:64] FLAG: --manifest-url=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498970    2580 flags.go:64] FLAG: --manifest-url-header=""
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498974    2580 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498977    2580 flags.go:64] FLAG: --max-open-files="1000000"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498981    2580 flags.go:64] FLAG: --max-pods="110"
Sep 16 14:07:57.505854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498984    2580 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498987    2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498990    2580 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498992    2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498995    2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.498998    2580 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499001    2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499008    2580 flags.go:64] FLAG: --node-status-max-images="50"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499011    2580 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499014    2580 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499017    2580 flags.go:64] FLAG: --pod-cidr=""
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499020    2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499026    2580 flags.go:64] FLAG: --pod-manifest-path=""
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499028    2580 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499045    2580 flags.go:64] FLAG: --pods-per-core="0"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499056    2580 flags.go:64] FLAG: --port="10250"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499059    2580 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499062    2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-079c89b854b200629"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499065    2580 flags.go:64] FLAG: --qos-reserved=""
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499068    2580 flags.go:64] FLAG: --read-only-port="10255"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499071    2580 flags.go:64] FLAG: --register-node="true"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499074    2580 flags.go:64] FLAG: --register-with-taints=""
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499082    2580 flags.go:64] FLAG: --registry-burst="10"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499085    2580 flags.go:64] FLAG: --registry-qps="5"
Sep 16 14:07:57.506505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499088    2580 flags.go:64] FLAG: --reserved-cpus=""
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499092    2580 flags.go:64] FLAG: --reserved-memory=""
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499095    2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499098    2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499102    2580 flags.go:64] FLAG: --rotate-certificates="false"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499105    2580 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499108    2580 flags.go:64] FLAG: --runonce="false"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499111    2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499114    2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499117    2580 flags.go:64] FLAG: --seccomp-default="false"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499120    2580 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499123    2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499126    2580 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499130    2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499133    2580 flags.go:64] FLAG: --storage-driver-password="root"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499135    2580 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499138    2580 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499141    2580 flags.go:64] FLAG: --storage-driver-user="root"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499144    2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499147    2580 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499150    2580 flags.go:64] FLAG: --system-cgroups=""
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499153    2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499159    2580 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499162    2580 flags.go:64] FLAG: --tls-cert-file=""
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499166    2580 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 16 14:07:57.507152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499171    2580 flags.go:64] FLAG: --tls-min-version=""
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499174    2580 flags.go:64] FLAG: --tls-private-key-file=""
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499177    2580 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499180    2580 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499183    2580 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499186    2580 flags.go:64] FLAG: --v="2"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499189    2580 flags.go:64] FLAG: --version="false"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499193    2580 flags.go:64] FLAG: --vmodule=""
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499198    2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.499201    2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499313    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499318    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499321    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499324    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499326    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499329    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499332    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499334    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499337    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499340    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499342    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499345    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.507786 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499348    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499350    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499353    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499355    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499358    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499360    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499363    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499367    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499370    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499372    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499376    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499378    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499381    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499384    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499386    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499389    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499391    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499394    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499398    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499400    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.508427 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499403    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499405    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499408    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499411    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499413    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499416    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499419    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499421    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499424    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499426    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499430    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499433    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499437    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499441    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499445    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499448    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499451    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499455    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499459    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.508917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499464    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499468    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499471    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499475    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499478    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499481    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499484    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499486    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499491    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499494    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499497    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499501    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499504    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499507    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499509    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499512    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499515    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499517    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499520    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499522    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.509402 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499525    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499528    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499530    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499533    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499535    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499538    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499541    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499543    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499546    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499548    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499551    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499554    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499558    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499560    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499563    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499565    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499568    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499571    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499573    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499576    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.509895 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499579    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499581    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499584    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499595    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499599    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499602    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499604    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.499607    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.500205    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.507481    2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.507496    2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.508846    2580 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.508855    2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508909    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.510406 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508914    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508917    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508921    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508924    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508928    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508931    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508934    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508937    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508939    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508942    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508945    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508948    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508950    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508953    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508956    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508958    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508961    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508964    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508967    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508969    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.510796 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508972    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508975    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508978    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508980    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508983    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508986    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508988    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508991    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508995    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.508998    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509000    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509003    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509006    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509009    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509012    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509014    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509017    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509020    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509022    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509025    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.511298 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509028    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509041    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509044    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509047    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509049    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509052    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509055    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509057    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509060    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509062    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509065    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509068    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509071    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509074    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509076    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509079    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509082    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509084    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509087    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509089    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.511783 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509092    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509096    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509099    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509101    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509104    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509107    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509109    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509112    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509115    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509117    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509121    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509125    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509128    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509130    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509133    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509135    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509138    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509140    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509143    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.512498 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509147    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509151    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509154    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509157    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509160    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509163    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509166    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509169    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509172    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509175    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509177    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509180    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509185    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509188    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509191    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509198    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509201    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509204    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.512955 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509207    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.509212    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509323    2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509328    2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509331    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509334    2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509337    2580 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509340    2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509343    2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509345    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509348    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509351    2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509353    2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509356    2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509359    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 16 14:07:57.513405 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509362    2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509364    2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509367    2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509369    2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509372    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509375    2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509377    2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509380    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509383    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509385    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509388    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509391    2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509394    2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509396    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509399    2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509402    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509405    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509408    2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509411    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509414    2580 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 16 14:07:57.513772 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509417    2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509420    2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509422    2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509426    2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509429    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509432    2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509435    2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509438    2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509441    2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509444    2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509447    2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509449    2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509452    2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509454    2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509457    2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509460    2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509464    2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509467    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509470    2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 16 14:07:57.514263 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509472    2580 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509475    2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509478    2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509482    2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509485    2580 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509488    2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509490    2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509493    2580 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509496    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509499    2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509501    2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509504    2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509506    2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509509    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509511    2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509514    2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509516    2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509519    2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509521    2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509524    2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 16 14:07:57.514723 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509526    2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509529    2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509531    2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509534    2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509536    2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509539    2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509541    2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509544    2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509546    2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509549    2580 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509552    2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509554    2580 feature_gate.go:328] unrecognized feature gate: Example2
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509557    2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509559    2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509562    2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509565    2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509567    2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509570    2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509573    2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509575    2580 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 16 14:07:57.515222 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509578    2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509580    2580 feature_gate.go:328] unrecognized feature gate: Example
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509583    2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509586    2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509588    2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509591    2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:07:57.509594    2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.509598    2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.509692    2580 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.511795    2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.512724    2580 server.go:1019] "Starting client certificate rotation"
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.512820    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:07:57.515704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.512855    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 16 14:07:57.516018 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.516001    2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:07:57.521407 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.521384    2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 16 14:07:57.532459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.532441    2580 log.go:25] "Validated CRI v1 runtime API"
Sep 16 14:07:57.535133 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.535118    2580 log.go:25] "Validated CRI v1 image API"
Sep 16 14:07:57.536741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.536725    2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 16 14:07:57.536916 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.536902    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:07:57.541384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.541362    2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 aaf72ff2-d2d2-4aec-b522-6255b3926779:/dev/nvme0n1p3 cfb90026-bd0e-4d36-b10c-52f03e943724:/dev/nvme0n1p4]
Sep 16 14:07:57.541445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.541383    2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 16 14:07:57.547104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.546988    2580 manager.go:216] Machine: {Timestamp:2026-09-16 14:07:57.545376212 +0000 UTC m=+0.321479695 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099464 MemoryCapacity:33164480512 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec297c4f413de59c6355f025fbd6bb1e SystemUUID:ec297c4f-413d-e59c-6355-f025fbd6bb1e BootID:c5aa385e-16e2-4a05-be2e-328a03d5ef1a Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582242304 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:e3:cc:8c:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:e3:cc:8c:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:d6:6f:ac:d8:7c:9a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164480512 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 16 14:07:57.547104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.547102    2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 16 14:07:57.547245 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.547206    2580 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 16 14:07:57.549989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.549967    2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 16 14:07:57.550147 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.549992    2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-80.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 16 14:07:57.550593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.550584    2580 topology_manager.go:138] "Creating topology manager with none policy"
Sep 16 14:07:57.550630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.550597    2580 container_manager_linux.go:309] "Creating device plugin manager"
Sep 16 14:07:57.550630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.550604    2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:07:57.551159 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.551148    2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 16 14:07:57.551858 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.551848    2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 16 14:07:57.554217 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554203    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:07:57.554336 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554326    2580 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 16 14:07:57.554416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554356    2580 kubelet.go:486] "Attempting to sync node with API server"
Sep 16 14:07:57.554416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554376    2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 16 14:07:57.554416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554393    2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 16 14:07:57.554416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554402    2580 kubelet.go:398] "Adding apiserver pod source"
Sep 16 14:07:57.554416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554410    2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 16 14:07:57.554822 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554810    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:07:57.554885 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.554832    2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 16 14:07:57.556856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.556840    2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 16 14:07:57.559224 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.559203    2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 16 14:07:57.559294 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.559253    2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 16 14:07:57.560432 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560407    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wcrc"
Sep 16 14:07:57.560856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560843    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 16 14:07:57.560905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560865    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 16 14:07:57.560905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560872    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 16 14:07:57.560905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560881    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 16 14:07:57.560905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560894    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 16 14:07:57.560905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560902    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560918    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560927    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560933    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560939    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560972    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 16 14:07:57.561056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.560979    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 16 14:07:57.561632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.561622    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 16 14:07:57.561662 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.561634    2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 16 14:07:57.565315 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.565297    2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-80.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 16 14:07:57.565668 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.565646    2580 server.go:1268] "Started kubelet"
Sep 16 14:07:57.565795 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.565770    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-80.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 16 14:07:57.565851 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.565773    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 16 14:07:57.565851 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.565819    2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 16 14:07:57.565925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.565878    2580 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 16 14:07:57.566045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.565919    2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 16 14:07:57.566560 ip-10-0-138-80 systemd[1]: Started Kubernetes Kubelet.
Sep 16 14:07:57.566985 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.566968    2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 16 14:07:57.573537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.571953    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wcrc"
Sep 16 14:07:57.573537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.573505    2580 server.go:310] "Adding debug handlers to kubelet server"
Sep 16 14:07:57.574830 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.574800    2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 16 14:07:57.577962 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.577943    2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 16 14:07:57.577962 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.577955    2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 16 14:07:57.578131 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.578015    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.578434 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.577383    2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-80.ec2.internal.18d5d2523ad1c8c1  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-80.ec2.internal,UID:ip-10-0-138-80.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-80.ec2.internal,},FirstTimestamp:2026-09-16 14:07:57.565618369 +0000 UTC m=+0.341721852,LastTimestamp:2026-09-16 14:07:57.565618369 +0000 UTC m=+0.341721852,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-80.ec2.internal,}"
Sep 16 14:07:57.578608 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578592    2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 16 14:07:57.578728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578691    2580 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 16 14:07:57.578728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578711    2580 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 16 14:07:57.578835 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578801    2580 reconstruct.go:97] "Volume reconstruction finished"
Sep 16 14:07:57.578835 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578811    2580 reconciler.go:29] "Reconciler: start to sync state"
Sep 16 14:07:57.578926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578893    2580 factory.go:55] Registering systemd factory
Sep 16 14:07:57.578926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.578907    2580 factory.go:223] Registration of the systemd container factory successfully
Sep 16 14:07:57.579168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579156    2580 factory.go:153] Registering CRI-O factory
Sep 16 14:07:57.579236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579170    2580 factory.go:223] Registration of the crio container factory successfully
Sep 16 14:07:57.579236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579221    2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 16 14:07:57.579236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579235    2580 factory.go:103] Registering Raw factory
Sep 16 14:07:57.579330 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579247    2580 manager.go:1196] Started watching for new ooms in manager
Sep 16 14:07:57.579652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.579641    2580 manager.go:349] Starting recovery of all containers
Sep 16 14:07:57.583701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.583678    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:07:57.586979 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.586959    2580 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-138-80.ec2.internal\" not found" node="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.590982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.590964    2580 manager.go:354] Recovery completed
Sep 16 14:07:57.595925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.595912    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.597848 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.597832    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.597921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.597864    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.597921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.597879    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.598567 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.598551    2580 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 16 14:07:57.598567 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.598565    2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 16 14:07:57.598717 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.598572    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.598717 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.598582    2580 state_mem.go:36] "Initialized new in-memory state store"
Sep 16 14:07:57.599403 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599387    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.599474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599417    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.599474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599434    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.599686 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599674    2580 policy_none.go:49] "None policy: Start"
Sep 16 14:07:57.599718 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599690    2580 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 16 14:07:57.599718 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.599700    2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 16 14:07:57.600697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.600686    2580 policy_none.go:47] "Start"
Sep 16 14:07:57.648833 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.648812    2580 manager.go:337] "Starting Device Plugin manager"
Sep 16 14:07:57.648957 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.648864    2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 16 14:07:57.648957 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.648879    2580 server.go:87] "Starting device plugin registration server"
Sep 16 14:07:57.649179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.649161    2580 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 16 14:07:57.649275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.649179    2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 16 14:07:57.649341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.649313    2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 16 14:07:57.649420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.649399    2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 16 14:07:57.649420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.649414    2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 16 14:07:57.649946 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.649924    2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 16 14:07:57.650028 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.649974    2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.664690 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.664669    2580 container_manager_linux.go:676] "Failed to ensure state" containerName="/system.slice" err="failed to move PID 2609 into the system container \"/system.slice\": <nil>"
Sep 16 14:07:57.749504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.749431    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.750448 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.750422    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.750571 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.750461    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.750571 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.750475    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.750571 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.750509    2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.757218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.757200    2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.771115 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.757230    2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-80.ec2.internal\": node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.778183 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.778163    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.806873 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.806843    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 16 14:07:57.807959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.807937    2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 16 14:07:57.808054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.807982    2580 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 16 14:07:57.808054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.808004    2580 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 16 14:07:57.808148 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.808068    2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 16 14:07:57.810521 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.810503    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:07:57.878855 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.878825    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.909080 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.909054    2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"]
Sep 16 14:07:57.909157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.909138    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.910836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.910816    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.910940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.910848    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.910940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.910862    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.912526 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.912513    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.912672 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.912657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.912709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.912691    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.913280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913266    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.913367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913297    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.913367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913312    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.913367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913271    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.913367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913358    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.913367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.913368    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.914688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.914675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.914739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.914699    2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 16 14:07:57.915387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.915361    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientMemory"
Sep 16 14:07:57.915469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.915392    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasNoDiskPressure"
Sep 16 14:07:57.915469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.915407    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeHasSufficientPID"
Sep 16 14:07:57.943762 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.943737    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-80.ec2.internal\" not found" node="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.946960 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.946944    2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-80.ec2.internal\" not found" node="ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.979261 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:57.979233    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:57.980365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.980345    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.980452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.980378    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:57.980452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:57.980407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a3ee3830b26f7f8f7ff158b9a7f8f57-config\") pod \"kube-apiserver-proxy-ip-10-0-138-80.ec2.internal\" (UID: \"2a3ee3830b26f7f8f7ff158b9a7f8f57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.080175 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:58.080090    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:58.081223 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a3ee3830b26f7f8f7ff158b9a7f8f57-config\") pod \"kube-apiserver-proxy-ip-10-0-138-80.ec2.internal\" (UID: \"2a3ee3830b26f7f8f7ff158b9a7f8f57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.081275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.081275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.081350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081333    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/2a3ee3830b26f7f8f7ff158b9a7f8f57-config\") pod \"kube-apiserver-proxy-ip-10-0-138-80.ec2.internal\" (UID: \"2a3ee3830b26f7f8f7ff158b9a7f8f57\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.081390 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081341    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.081419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.081382    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/50a8e81bafdf533f0380af356a5122a5-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal\" (UID: \"50a8e81bafdf533f0380af356a5122a5\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:58.180513 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:58.180475    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:58.246742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:58.246715    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:59.620247 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.619912    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:59.650622 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:59.650592    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:59.751149 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:59.751051    2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-80.ec2.internal\" not found"
Sep 16 14:07:59.772656 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.772627    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:07:59.848301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.848265    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:59.861191 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.861170    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:07:59.863314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.863302    2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal"
Sep 16 14:07:59.872794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.872779    2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 16 14:07:59.881928 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.881910    2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 16 14:07:59.882043 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.882024    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:07:59.882086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.882025    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:07:59.882086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.882025    2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 16 14:07:59.924167 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.924145    2580 apiserver.go:52] "Watching apiserver"
Sep 16 14:07:59.937707 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.937683    2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 16 14:07:59.939260 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.939237    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-zgc2b","kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8","openshift-cluster-node-tuning-operator/tuned-sdgfr","openshift-dns/node-resolver-kz8kh","openshift-multus/multus-additional-cni-plugins-mh2p9","openshift-multus/multus-mvj4t","openshift-image-registry/node-ca-lgpt4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal","openshift-multus/network-metrics-daemon-xtwhx","openshift-network-diagnostics/network-check-target-xx49p","openshift-network-operator/iptables-alerter-7nplm","openshift-ovn-kubernetes/ovnkube-node-jgppg","kube-system/global-pull-secret-syncer-klv79"]
Sep 16 14:07:59.940690 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.940657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:07:59.941801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.941785    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.942654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.942633    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.943372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.943359    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 16 14:07:59.943749 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.943733    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 16 14:07:59.943827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.943813    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:07:59.943948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.943934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-c9z8x\""
Sep 16 14:07:59.944261 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.944246    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.945256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.944882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 16 14:07:59.945256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.944999    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.945256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.945031    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-8zrgz\""
Sep 16 14:07:59.945256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.945100    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9hfbg\""
Sep 16 14:07:59.945596 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.945577    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.945811 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.945796    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.947251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.946962    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-15 14:02:58 +0000 UTC" deadline="2028-04-26 18:36:28.659618231 +0000 UTC"
Sep 16 14:07:59.947342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.947253    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14116h28m28.71237087s"
Sep 16 14:07:59.947342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.947260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.947577 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.947536    2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 16 14:07:59.948349 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.948335    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.949318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.949300    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:07:59.950704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.950691    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:07:59.950767 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:59.950751    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:07:59.952030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952017    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:07:59.952076 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:59.952066    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:07:59.952225 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952210    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-hostroot\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-daemon-config\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952278    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kfh6h\" (UniqueName: \"kubernetes.io/projected/53838c2b-0d4c-4313-89f4-28728b38e7c2-kube-api-access-kfh6h\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952303    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kmc98\" (UniqueName: \"kubernetes.io/projected/336f426e-189a-4e10-a4cc-05fcc62e2a58-kube-api-access-kmc98\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.952379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952319    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-conf\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952334    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-systemd\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-cnibin\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952382    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-cnibin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952400    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-cni-binary-copy\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952414    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-sys-fs\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952432    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-modprobe-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952453    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-kubernetes\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952472    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-run\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952493    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952578    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-k8s-cni-cncf-io\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952607    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-netns\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952624    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-bin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-system-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-multus-certs\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-etc-kubernetes\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952755    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-multus\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952809    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-var-lib-kubelet\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.952847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952830    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-host\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952844    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-tuned\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952887    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n7z6z\" (UniqueName: \"kubernetes.io/projected/82f124ab-49cc-4b45-8eac-344f52d7b8f8-kube-api-access-n7z6z\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c0520e-773d-4d35-8f4a-394f677fea8d-tmp-dir\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952931    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-device-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.952989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-etc-selinux\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953055    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953079    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-sys\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953117    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-79pc9\" (UniqueName: \"kubernetes.io/projected/a1c0520e-773d-4d35-8f4a-394f677fea8d-kube-api-access-79pc9\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953144    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-socket-dir-parent\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-kubelet\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953209    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-conf-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3029ea23-b949-46e9-8b79-03ffcbf38066-konnectivity-ca\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953255    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-socket-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.953297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-lib-modules\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953311    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-tmp\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953340    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c0520e-773d-4d35-8f4a-394f677fea8d-hosts-file\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953399    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953428    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cfk\" (UniqueName: \"kubernetes.io/projected/bee02a73-e799-45f1-88c1-91c30f3428c0-kube-api-access-d5cfk\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953460    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3029ea23-b949-46e9-8b79-03ffcbf38066-agent-certs\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953485    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-registration-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953516    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysconfig\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953539    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953613    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953624    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953663    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953676    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953624    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953762    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-p4grs\""
Sep 16 14:07:59.954045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.953814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 16 14:07:59.954673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.954331    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-vb8p8\""
Sep 16 14:07:59.954673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.954479    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:07:59.954673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.954551    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 16 14:07:59.954673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.954656    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 16 14:07:59.955009 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.954989    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-gnrpc\""
Sep 16 14:07:59.955104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.955033    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.955104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.955038    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.955213 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.955111    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-vcqfc\""
Sep 16 14:07:59.955801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.955780    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:07:59.955869 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:07:59.955851    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:07:59.956616 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956600    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.956688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956605    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.956825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956810    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 16 14:07:59.956873 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956829    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 16 14:07:59.956873 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956831    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 16 14:07:59.956949 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956848    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 16 14:07:59.956949 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.956913    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 16 14:07:59.957064 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.957053    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 16 14:07:59.957108 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.957066    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-rqqx6\""
Sep 16 14:07:59.958932 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.958917    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 16 14:07:59.959060 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.959044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wqqt2\""
Sep 16 14:07:59.965855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.965837    2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 16 14:07:59.990726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.990699    2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-jbfl8"
Sep 16 14:07:59.999228 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:07:59.999206    2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-jbfl8"
Sep 16 14:08:00.049660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.049638    2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 16 14:08:00.054425 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054399    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-kubelet\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-serviceca\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054467    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-var-lib-kubelet\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054488    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n7z6z\" (UniqueName: \"kubernetes.io/projected/82f124ab-49cc-4b45-8eac-344f52d7b8f8-kube-api-access-n7z6z\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054509    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c0520e-773d-4d35-8f4a-394f677fea8d-tmp-dir\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054516    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-kubelet\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054531    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-device-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-var-lib-kubelet\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054584    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-etc-selinux\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054598    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-device-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054607    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-conf-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-conf-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.054611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054637    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4jbg\" (UniqueName: \"kubernetes.io/projected/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-kube-api-access-s4jbg\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054665    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-systemd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-etc-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054714    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-ovn\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054724    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-etc-selinux\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054744    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3029ea23-b949-46e9-8b79-03ffcbf38066-konnectivity-ca\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054790    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-socket-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054830    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-lib-modules\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-tmp\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c0520e-773d-4d35-8f4a-394f677fea8d-hosts-file\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054946    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-lib-modules\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054948    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054991    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cfk\" (UniqueName: \"kubernetes.io/projected/bee02a73-e799-45f1-88c1-91c30f3428c0-kube-api-access-d5cfk\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055000    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a1c0520e-773d-4d35-8f4a-394f677fea8d-hosts-file\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.054951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a1c0520e-773d-4d35-8f4a-394f677fea8d-tmp-dir\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055020    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-kubelet\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055048    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3029ea23-b949-46e9-8b79-03ffcbf38066-agent-certs\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:00.055297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysconfig\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055106    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-hostroot\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055135    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kfh6h\" (UniqueName: \"kubernetes.io/projected/53838c2b-0d4c-4313-89f4-28728b38e7c2-kube-api-access-kfh6h\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055159    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79fa1313-3cd2-4310-a282-501928420a95-ovn-node-metrics-cert\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055188    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-conf\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055211    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a942c87-447f-4ff1-9420-df5cc9e4318f-iptables-alerter-script\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-run\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055257    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-bin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055252    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/3029ea23-b949-46e9-8b79-03ffcbf38066-konnectivity-ca\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-hostroot\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055280    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-netns\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055376    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-system-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055445    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-bin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055491    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysconfig\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-conf\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-run\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:00.056195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055789    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-var-lib-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-script-lib\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055825    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-system-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.055954    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-netd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-79pc9\" (UniqueName: \"kubernetes.io/projected/a1c0520e-773d-4d35-8f4a-394f677fea8d-kube-api-access-79pc9\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056123    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-host\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056109    2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056159    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-tuned\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-host\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056398    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-socket-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056509    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-binary-copy\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056574    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056601    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-dbus\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.057128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056636    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056660    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-socket-dir-parent\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056684    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-host\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056700    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-kubelet-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056726    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bsm2\" (UniqueName: \"kubernetes.io/projected/8a942c87-447f-4ff1-9420-df5cc9e4318f-kube-api-access-2bsm2\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056750    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-systemd-units\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-system-cni-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-socket-dir-parent\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056865    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-slash\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056913    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-log-socket\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056941    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.056970    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d7kd8\" (UniqueName: \"kubernetes.io/projected/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-kube-api-access-d7kd8\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-bin\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057196    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-env-overrides\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.057890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057205    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057218    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-kubelet-config\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-registration-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057279    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-daemon-config\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057324    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a942c87-447f-4ff1-9420-df5cc9e4318f-host-slash\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057348    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kmc98\" (UniqueName: \"kubernetes.io/projected/336f426e-189a-4e10-a4cc-05fcc62e2a58-kube-api-access-kmc98\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057384    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-systemd\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057408    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-cnibin\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057434    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-cnibin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057459    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-cni-binary-copy\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057482    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-node-log\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057506    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-sys-fs\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057529    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-modprobe-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-kubernetes\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057595    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-k8s-cni-cncf-io\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.058867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057664    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-netns\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-k8s-cni-cncf-io\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057693    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-etc-kubernetes\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057721    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zsxn4\" (UniqueName: \"kubernetes.io/projected/79fa1313-3cd2-4310-a282-501928420a95-kube-api-access-zsxn4\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-registration-dir\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-multus-certs\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057780    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-netns\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-config\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057824    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-run-multus-certs\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057836    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-multus\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057859    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057861    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-etc-kubernetes\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057882    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-sys\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057957    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-systemd\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-sys\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.057993    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bee02a73-e799-45f1-88c1-91c30f3428c0-cnibin\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058010    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-host-var-lib-cni-multus\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.059761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-cnibin\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058080    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-sysctl-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058160    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-modprobe-d\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058220    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/336f426e-189a-4e10-a4cc-05fcc62e2a58-sys-fs\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-kubernetes\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058327    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/bee02a73-e799-45f1-88c1-91c30f3428c0-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-cni-dir\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-multus-daemon-config\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.058685    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/53838c2b-0d4c-4313-89f4-28728b38e7c2-cni-binary-copy\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.059997    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-tmp\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.060201    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/3029ea23-b949-46e9-8b79-03ffcbf38066-agent-certs\") pod \"konnectivity-agent-zgc2b\" (UID: \"3029ea23-b949-46e9-8b79-03ffcbf38066\") " pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:00.060609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.060262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/82f124ab-49cc-4b45-8eac-344f52d7b8f8-etc-tuned\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.063433 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.063414    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n7z6z\" (UniqueName: \"kubernetes.io/projected/82f124ab-49cc-4b45-8eac-344f52d7b8f8-kube-api-access-n7z6z\") pod \"tuned-sdgfr\" (UID: \"82f124ab-49cc-4b45-8eac-344f52d7b8f8\") " pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.063537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.063522    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kfh6h\" (UniqueName: \"kubernetes.io/projected/53838c2b-0d4c-4313-89f4-28728b38e7c2-kube-api-access-kfh6h\") pod \"multus-mvj4t\" (UID: \"53838c2b-0d4c-4313-89f4-28728b38e7c2\") " pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.063711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.063695    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cfk\" (UniqueName: \"kubernetes.io/projected/bee02a73-e799-45f1-88c1-91c30f3428c0-kube-api-access-d5cfk\") pod \"multus-additional-cni-plugins-mh2p9\" (UID: \"bee02a73-e799-45f1-88c1-91c30f3428c0\") " pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.069485 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.069461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kmc98\" (UniqueName: \"kubernetes.io/projected/336f426e-189a-4e10-a4cc-05fcc62e2a58-kube-api-access-kmc98\") pod \"aws-ebs-csi-driver-node-sk9m8\" (UID: \"336f426e-189a-4e10-a4cc-05fcc62e2a58\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.069759 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.069742    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-79pc9\" (UniqueName: \"kubernetes.io/projected/a1c0520e-773d-4d35-8f4a-394f677fea8d-kube-api-access-79pc9\") pod \"node-resolver-kz8kh\" (UID: \"a1c0520e-773d-4d35-8f4a-394f677fea8d\") " pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.076603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.076583    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mvj4t"
Sep 16 14:08:00.155773 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.155745    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50a8e81bafdf533f0380af356a5122a5.slice/crio-053535cff8b642c265893e229744fba3d69f7435904e4cdfc1e5762cf576084c WatchSource:0}: Error finding container 053535cff8b642c265893e229744fba3d69f7435904e4cdfc1e5762cf576084c: Status 404 returned error can't find the container with id 053535cff8b642c265893e229744fba3d69f7435904e4cdfc1e5762cf576084c
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158281    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-host\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158330    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bsm2\" (UniqueName: \"kubernetes.io/projected/8a942c87-447f-4ff1-9420-df5cc9e4318f-kube-api-access-2bsm2\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-systemd-units\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-slash\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158467    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-host\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-log-socket\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-systemd-units\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158510    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.158540    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a3ee3830b26f7f8f7ff158b9a7f8f57.slice/crio-3428fd1612ec0595dba4f25cfe3e709f73b4a16f7c865e4201d4b3e61b23696b WatchSource:0}: Error finding container 3428fd1612ec0595dba4f25cfe3e709f73b4a16f7c865e4201d4b3e61b23696b: Status 404 returned error can't find the container with id 3428fd1612ec0595dba4f25cfe3e709f73b4a16f7c865e4201d4b3e61b23696b
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158580    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d7kd8\" (UniqueName: \"kubernetes.io/projected/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-kube-api-access-d7kd8\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.158601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158592    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-slash\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158648    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-log-socket\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158699    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158747    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-bin\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158792    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-bin\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-env-overrides\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158872    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-kubelet-config\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.158924    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a942c87-447f-4ff1-9420-df5cc9e4318f-host-slash\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-node-log\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159055    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zsxn4\" (UniqueName: \"kubernetes.io/projected/79fa1313-3cd2-4310-a282-501928420a95-kube-api-access-zsxn4\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-config\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159126    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-serviceca\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s4jbg\" (UniqueName: \"kubernetes.io/projected/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-kube-api-access-s4jbg\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.159249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-systemd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159292    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-node-log\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-env-overrides\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-etc-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.159350    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159374    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-ovn\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-kubelet-config\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-kubelet\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.159449    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:00.659406103 +0000 UTC m=+2.065986308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79fa1313-3cd2-4310-a282-501928420a95-ovn-node-metrics-cert\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159574    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a942c87-447f-4ff1-9420-df5cc9e4318f-iptables-alerter-script\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159576    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-kubelet\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/8a942c87-447f-4ff1-9420-df5cc9e4318f-host-slash\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159673    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-netns\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.159983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159723    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-etc-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160143    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-serviceca\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-config\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.159612    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-run-netns\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160593    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-var-lib-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-script-lib\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160652    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160681    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-netd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.160720 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160717    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160746    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-dbus\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160773    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-ovn\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/8a942c87-447f-4ff1-9420-df5cc9e4318f-iptables-alerter-script\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160845    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-dbus\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160906    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-cni-netd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.160938    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.160951    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-var-lib-openvswitch\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161124 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.160992    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:00.660968593 +0000 UTC m=+2.067548794 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:00.161545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.161340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/79fa1313-3cd2-4310-a282-501928420a95-ovnkube-script-lib\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.161380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.161545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.161400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/79fa1313-3cd2-4310-a282-501928420a95-run-systemd\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.162300 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.162261    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:08:00.163279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.163260    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/79fa1313-3cd2-4310-a282-501928420a95-ovn-node-metrics-cert\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.168325 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.168286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bsm2\" (UniqueName: \"kubernetes.io/projected/8a942c87-447f-4ff1-9420-df5cc9e4318f-kube-api-access-2bsm2\") pod \"iptables-alerter-7nplm\" (UID: \"8a942c87-447f-4ff1-9420-df5cc9e4318f\") " pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.168470 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.168455    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:00.168588 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.168475    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:00.168588 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.168488    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:00.168588 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.168572    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:00.66853314 +0000 UTC m=+2.075113327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:00.169806 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.169777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d7kd8\" (UniqueName: \"kubernetes.io/projected/17ba2d77-eb74-4d56-a93f-7df4d41b2ac5-kube-api-access-d7kd8\") pod \"node-ca-lgpt4\" (UID: \"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5\") " pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.170457 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.170441    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4jbg\" (UniqueName: \"kubernetes.io/projected/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-kube-api-access-s4jbg\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.170590 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.170575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zsxn4\" (UniqueName: \"kubernetes.io/projected/79fa1313-3cd2-4310-a282-501928420a95-kube-api-access-zsxn4\") pod \"ovnkube-node-jgppg\" (UID: \"79fa1313-3cd2-4310-a282-501928420a95\") " pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.180489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.180437    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal" event={"ID":"2a3ee3830b26f7f8f7ff158b9a7f8f57","Type":"ContainerStarted","Data":"3428fd1612ec0595dba4f25cfe3e709f73b4a16f7c865e4201d4b3e61b23696b"}
Sep 16 14:08:00.181390 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.181374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal" event={"ID":"50a8e81bafdf533f0380af356a5122a5","Type":"ContainerStarted","Data":"053535cff8b642c265893e229744fba3d69f7435904e4cdfc1e5762cf576084c"}
Sep 16 14:08:00.274807 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.274735    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:00.281330 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.281309    2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:00.283099 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.283077    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3029ea23_b949_46e9_8b79_03ffcbf38066.slice/crio-2947e0bbd7c383017742a63fbfea1e8fd0f6674288ed24b96637f3b5036f2daf WatchSource:0}: Error finding container 2947e0bbd7c383017742a63fbfea1e8fd0f6674288ed24b96637f3b5036f2daf: Status 404 returned error can't find the container with id 2947e0bbd7c383017742a63fbfea1e8fd0f6674288ed24b96637f3b5036f2daf
Sep 16 14:08:00.289748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.289731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8"
Sep 16 14:08:00.296744 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.296722    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod336f426e_189a_4e10_a4cc_05fcc62e2a58.slice/crio-5208901ed6dc706240bc3423a24438800807b7d63bf920ca5de98882eee47d47 WatchSource:0}: Error finding container 5208901ed6dc706240bc3423a24438800807b7d63bf920ca5de98882eee47d47: Status 404 returned error can't find the container with id 5208901ed6dc706240bc3423a24438800807b7d63bf920ca5de98882eee47d47
Sep 16 14:08:00.309746 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.309728    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sdgfr"
Sep 16 14:08:00.316485 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.316463    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod82f124ab_49cc_4b45_8eac_344f52d7b8f8.slice/crio-eb866451f9139814f4168bcd465a52616176e6f09fdba9fd068c515c00920802 WatchSource:0}: Error finding container eb866451f9139814f4168bcd465a52616176e6f09fdba9fd068c515c00920802: Status 404 returned error can't find the container with id eb866451f9139814f4168bcd465a52616176e6f09fdba9fd068c515c00920802
Sep 16 14:08:00.330649 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.330628    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-kz8kh"
Sep 16 14:08:00.336757 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.336740    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mh2p9"
Sep 16 14:08:00.337007 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.336990    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1c0520e_773d_4d35_8f4a_394f677fea8d.slice/crio-20435acc2fb501f288915a3e08d5b56e3315ab90b33b5cc73628276837936d2f WatchSource:0}: Error finding container 20435acc2fb501f288915a3e08d5b56e3315ab90b33b5cc73628276837936d2f: Status 404 returned error can't find the container with id 20435acc2fb501f288915a3e08d5b56e3315ab90b33b5cc73628276837936d2f
Sep 16 14:08:00.343725 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.343704    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbee02a73_e799_45f1_88c1_91c30f3428c0.slice/crio-daae90c364aa92a940ab153b7b7a02f9a6736fcfd68520d83cb6e87ebadc1c19 WatchSource:0}: Error finding container daae90c364aa92a940ab153b7b7a02f9a6736fcfd68520d83cb6e87ebadc1c19: Status 404 returned error can't find the container with id daae90c364aa92a940ab153b7b7a02f9a6736fcfd68520d83cb6e87ebadc1c19
Sep 16 14:08:00.391474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.391450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-lgpt4"
Sep 16 14:08:00.398388 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.398366    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17ba2d77_eb74_4d56_a93f_7df4d41b2ac5.slice/crio-1a8d1484b3a3682c447abad8f03afc33da5d319b4d17ed434457cc4e0a712afe WatchSource:0}: Error finding container 1a8d1484b3a3682c447abad8f03afc33da5d319b4d17ed434457cc4e0a712afe: Status 404 returned error can't find the container with id 1a8d1484b3a3682c447abad8f03afc33da5d319b4d17ed434457cc4e0a712afe
Sep 16 14:08:00.402342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.402326    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7nplm"
Sep 16 14:08:00.408099 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.408078    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:00.409282 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.409261    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8a942c87_447f_4ff1_9420_df5cc9e4318f.slice/crio-72cb17eca0bba9f01632bf99c5f1443de4d7edecbc867c6585437aa4faff2b23 WatchSource:0}: Error finding container 72cb17eca0bba9f01632bf99c5f1443de4d7edecbc867c6585437aa4faff2b23: Status 404 returned error can't find the container with id 72cb17eca0bba9f01632bf99c5f1443de4d7edecbc867c6585437aa4faff2b23
Sep 16 14:08:00.415081 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.415062    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79fa1313_3cd2_4310_a282_501928420a95.slice/crio-0bec604c26f658080a43a37490ebe0afcb587c118c39ce0b6fa64371e706594d WatchSource:0}: Error finding container 0bec604c26f658080a43a37490ebe0afcb587c118c39ce0b6fa64371e706594d: Status 404 returned error can't find the container with id 0bec604c26f658080a43a37490ebe0afcb587c118c39ce0b6fa64371e706594d
Sep 16 14:08:00.459485 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:00.459458    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53838c2b_0d4c_4313_89f4_28728b38e7c2.slice/crio-c1b829e1d9c363cbdbde706c9788aa55c73954bb30b61cf67c8b48b89c30a661 WatchSource:0}: Error finding container c1b829e1d9c363cbdbde706c9788aa55c73954bb30b61cf67c8b48b89c30a661: Status 404 returned error can't find the container with id c1b829e1d9c363cbdbde706c9788aa55c73954bb30b61cf67c8b48b89c30a661
Sep 16 14:08:00.664210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.664114    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:00.664210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.664204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:00.664418 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.664272    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:00.664418 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.664337    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:01.664321032 +0000 UTC m=+3.070901223 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:00.664418 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.664339    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:00.664418 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.664392    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:01.664375394 +0000 UTC m=+3.070955586 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:00.765062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.765027    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:00.765236 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.765156    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:00.765236 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.765173    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:00.765236 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.765182    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:00.765362 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:00.765246    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:01.765226379 +0000 UTC m=+3.171806585 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:00.810659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:00.810624    2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:01.001543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.001360    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 14:02:59 +0000 UTC" deadline="2028-04-21 13:49:46.340129153 +0000 UTC"
Sep 16 14:08:01.001543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.001424    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13991h41m45.338709557s"
Sep 16 14:08:01.179236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.178668    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:01.179236 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.178794    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:01.185132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.185040    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"0bec604c26f658080a43a37490ebe0afcb587c118c39ce0b6fa64371e706594d"}
Sep 16 14:08:01.188319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.188291    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgpt4" event={"ID":"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5","Type":"ContainerStarted","Data":"1a8d1484b3a3682c447abad8f03afc33da5d319b4d17ed434457cc4e0a712afe"}
Sep 16 14:08:01.190147 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.190116    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerStarted","Data":"daae90c364aa92a940ab153b7b7a02f9a6736fcfd68520d83cb6e87ebadc1c19"}
Sep 16 14:08:01.197803 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.197778    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz8kh" event={"ID":"a1c0520e-773d-4d35-8f4a-394f677fea8d","Type":"ContainerStarted","Data":"20435acc2fb501f288915a3e08d5b56e3315ab90b33b5cc73628276837936d2f"}
Sep 16 14:08:01.201089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.201062    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdgfr" event={"ID":"82f124ab-49cc-4b45-8eac-344f52d7b8f8","Type":"ContainerStarted","Data":"eb866451f9139814f4168bcd465a52616176e6f09fdba9fd068c515c00920802"}
Sep 16 14:08:01.202973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.202949    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8" event={"ID":"336f426e-189a-4e10-a4cc-05fcc62e2a58","Type":"ContainerStarted","Data":"5208901ed6dc706240bc3423a24438800807b7d63bf920ca5de98882eee47d47"}
Sep 16 14:08:01.204980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.204957    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mvj4t" event={"ID":"53838c2b-0d4c-4313-89f4-28728b38e7c2","Type":"ContainerStarted","Data":"c1b829e1d9c363cbdbde706c9788aa55c73954bb30b61cf67c8b48b89c30a661"}
Sep 16 14:08:01.206504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.206482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7nplm" event={"ID":"8a942c87-447f-4ff1-9420-df5cc9e4318f","Type":"ContainerStarted","Data":"72cb17eca0bba9f01632bf99c5f1443de4d7edecbc867c6585437aa4faff2b23"}
Sep 16 14:08:01.209245 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.209222    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgc2b" event={"ID":"3029ea23-b949-46e9-8b79-03ffcbf38066","Type":"ContainerStarted","Data":"2947e0bbd7c383017742a63fbfea1e8fd0f6674288ed24b96637f3b5036f2daf"}
Sep 16 14:08:01.435512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.435430    2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:01.672305 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.672270    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:01.672535 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.672344    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:01.672535 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.672470    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:01.672535 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.672536    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.672517166 +0000 UTC m=+5.079097376 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:01.673034 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.673015    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:01.673144 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.673079    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.673063759 +0000 UTC m=+5.079643951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:01.773326 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:01.773222    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:01.773487 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.773471    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:01.773574 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.773491    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:01.773574 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.773504    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:01.773691 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:01.773577    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:03.773542732 +0000 UTC m=+5.180122921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:02.001640 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:02.001578    2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-15 14:02:59 +0000 UTC" deadline="2028-03-26 16:04:37.387785459 +0000 UTC"
Sep 16 14:08:02.001640 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:02.001636    2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13369h56m35.386153627s"
Sep 16 14:08:02.178084 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:02.178005    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:02.178251 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:02.178132    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:02.178251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:02.178161    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:02.178251 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:02.178245    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:02.559452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:02.559174    2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 16 14:08:03.179056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:03.179021    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:03.179527 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.179160    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:03.687869 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:03.687829    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:03.688059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:03.687914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:03.688129 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.688072    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.688184 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.688140    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:07.68811894 +0000 UTC m=+9.094699132 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:03.688578 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.688534    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:03.688672 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.688612    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:07.688593616 +0000 UTC m=+9.095173806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:03.789007 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:03.788966    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:03.789207 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.789173    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:03.789207 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.789189    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:03.789207 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.789201    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:03.789372 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:03.789261    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:07.789246957 +0000 UTC m=+9.195827163 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:04.178538 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:04.178505    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:04.178691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:04.178497    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:04.178759 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:04.178689    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:04.178860 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:04.178812    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:05.179998 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:05.179969    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:05.180654 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:05.180096    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:06.178252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:06.178212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:06.178252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:06.178258    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:06.178516 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:06.178341    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:06.178516 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:06.178491    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:07.178849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:07.178776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:07.179319 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.178922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:07.722280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:07.722220    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:07.722452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:07.722302    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:07.722452 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.722390    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:07.722452 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.722416    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:07.722623 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.722460    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:15.722440414 +0000 UTC m=+17.129020605 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:07.722623 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.722481    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:15.72247077 +0000 UTC m=+17.129050962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:07.823440 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:07.823400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:07.823753 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.823671    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:07.823753 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.823699    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:07.823753 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.823712    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:07.823912 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:07.823778    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:15.823759255 +0000 UTC m=+17.230339447 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:08.178364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:08.178323    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:08.178575 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:08.178451    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:08.178659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:08.178637    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:08.178787 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:08.178748    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:09.179035 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:09.178996    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:09.179492 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:09.179145    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:10.178653 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:10.178470    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:10.178653 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:10.178614    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:10.178873 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:10.178669    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:10.178931 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:10.178866    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:11.178376 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:11.178340    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:11.178810 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:11.178493    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:12.178474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:12.178434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:12.178949 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:12.178434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:12.178949 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:12.178583    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:12.178949 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:12.178713    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:13.178468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:13.178432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:13.178677 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:13.178580    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:14.178798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:14.178756    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:14.179258 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:14.178896    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:14.179258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:14.178940    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:14.179258 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:14.179029    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:15.178258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:15.178212    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:15.178433 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.178347    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:15.780938 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:15.780902    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:15.781398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:15.780972    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:15.781398 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.781060    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:15.781398 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.781088    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:15.781398 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.781146    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:31.781123653 +0000 UTC m=+33.187703853 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:15.781398 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.781166    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:08:31.781156345 +0000 UTC m=+33.187736534 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:15.881581 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:15.881539    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:15.881737 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.881690    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:15.881737 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.881705    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:15.881737 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.881714    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:15.881866 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:15.881760    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:31.881747522 +0000 UTC m=+33.288327708 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:16.177921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:16.177854    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:16.177921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:16.177892    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:16.178134 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:16.177961    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:16.178134 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:16.178084    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:17.177847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:17.177809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:17.178327 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:17.177935    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:18.178133 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:18.178094    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:18.178604 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:18.178219    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:18.178604 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:18.178107    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:18.178604 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:18.178578    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:19.179243 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.178704    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:19.179243 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:19.179201    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:19.251809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.251774    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"7a408194f393a4c8d572fded4961b897197e943d3c600b1f30c134220a74d17d"}
Sep 16 14:08:19.251909 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.251873    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"ef7f1f0ec7b348c825907f3d21c2c5cb68f6201b5483514de866f3e78ce2c652"}
Sep 16 14:08:19.251963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.251928    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"4f12f2c6a4f8dd698e5da2d7125701620c31602afc9f421eafaa8fed798cf79d"}
Sep 16 14:08:19.252024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.251972    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"3688ee7b8d6752d93acba07e2092c8010a3ad9446bfd1a9a465baaa5876bcbb8"}
Sep 16 14:08:19.252024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.252004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"3719b95e2a33931f68c51dc5896d22f025f6fe8379d4e403d247e7c248889bcb"}
Sep 16 14:08:19.255497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.255465    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-lgpt4" event={"ID":"17ba2d77-eb74-4d56-a93f-7df4d41b2ac5","Type":"ContainerStarted","Data":"67c10edeb7316afc95ecc3efcbbbcf21c8bd95c491f4f89ae33f331f8b035a89"}
Sep 16 14:08:19.257177 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.257149    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerStarted","Data":"001f3b433d82a25a360fb552d372de586cc1b1aa4c9696df1aacea0870ac1fbb"}
Sep 16 14:08:19.258987 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.258969    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-kz8kh" event={"ID":"a1c0520e-773d-4d35-8f4a-394f677fea8d","Type":"ContainerStarted","Data":"21290f34446a918ba21a18550fd8ffd0c1e6647b342945f2c06d9903f9753437"}
Sep 16 14:08:19.260624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.260585    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sdgfr" event={"ID":"82f124ab-49cc-4b45-8eac-344f52d7b8f8","Type":"ContainerStarted","Data":"a60e7bffd27dc89ae7c4763affb0a86f04c675403fb3db7fdf14130b4d7397d9"}
Sep 16 14:08:19.261897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.261878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8" event={"ID":"336f426e-189a-4e10-a4cc-05fcc62e2a58","Type":"ContainerStarted","Data":"75df3cb89b00b670a152a8ba15a5d54e0253d7201ea42d6e40b9727176358dda"}
Sep 16 14:08:19.263412 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.263393    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal" event={"ID":"2a3ee3830b26f7f8f7ff158b9a7f8f57","Type":"ContainerStarted","Data":"4971dbde5c8f07728426a36161fd49e606ad3b6bb55a476d9a04f550978afb3a"}
Sep 16 14:08:19.265285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.265064    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mvj4t" event={"ID":"53838c2b-0d4c-4313-89f4-28728b38e7c2","Type":"ContainerStarted","Data":"66b13c9b009c5e23da51fed0afe0749dcf60905c6f08df3d080a50ad756170ee"}
Sep 16 14:08:19.266971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.266838    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-zgc2b" event={"ID":"3029ea23-b949-46e9-8b79-03ffcbf38066","Type":"ContainerStarted","Data":"83a69acd6fd1d67f2de6d6a3395666a57ddd92fda43570c4457f16f07bff7c12"}
Sep 16 14:08:19.269964 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.269908    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-lgpt4" podStartSLOduration=1.9124965440000001 podStartE2EDuration="20.269894255s" totalImagesPullingTime="18.357397711s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.26986851 +0000 UTC m=+20.676448719" watchObservedRunningTime="2026-09-16 14:08:19.269894255 +0000 UTC m=+20.676474463"
Sep 16 14:08:19.285752 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.285716    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-zgc2b" podStartSLOduration=9.880747869 podStartE2EDuration="20.285704326s" totalImagesPullingTime="10.404956457s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.284914764 +0000 UTC m=+20.691494983" watchObservedRunningTime="2026-09-16 14:08:19.285704326 +0000 UTC m=+20.692284534"
Sep 16 14:08:19.300893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.300856    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sdgfr" podStartSLOduration=1.949057679 podStartE2EDuration="20.30084429s" totalImagesPullingTime="18.351786611s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.300600611 +0000 UTC m=+20.707180819" watchObservedRunningTime="2026-09-16 14:08:19.30084429 +0000 UTC m=+20.707424498"
Sep 16 14:08:19.334164 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.334126    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-80.ec2.internal" podStartSLOduration=20.334114395 podStartE2EDuration="20.334114395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.333812009 +0000 UTC m=+20.740392216" watchObservedRunningTime="2026-09-16 14:08:19.334114395 +0000 UTC m=+20.740694602"
Sep 16 14:08:19.381456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.381407    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-kz8kh" podStartSLOduration=1.9624554669999998 podStartE2EDuration="20.381388355s" totalImagesPullingTime="18.418932888s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.351317018 +0000 UTC m=+20.757897225" watchObservedRunningTime="2026-09-16 14:08:19.381388355 +0000 UTC m=+20.787968563"
Sep 16 14:08:19.381649 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:19.381623    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-mvj4t" podStartSLOduration=2.08484479 podStartE2EDuration="20.381611312s" totalImagesPullingTime="18.296766522s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:19.380433578 +0000 UTC m=+20.787013768" watchObservedRunningTime="2026-09-16 14:08:19.381611312 +0000 UTC m=+20.788191523"
Sep 16 14:08:20.023097 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.022951    2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 16 14:08:20.029837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.029761    2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-16T14:08:20.023092044Z","UUID":"6194961b-4cd7-4db7-a420-1337554a0887","Handler":null,"Name":"","Endpoint":""}
Sep 16 14:08:20.030995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.030977    2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 16 14:08:20.031091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.031010    2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 16 14:08:20.178765 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.178739    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:20.178919 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:20.178849    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:20.178919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.178860    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:20.179002 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:20.178961    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:20.270373 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.270342    2580 generic.go:356] "Generic (PLEG): container finished" podID="50a8e81bafdf533f0380af356a5122a5" containerID="34a5ae4bc2ca62181584ed1fef8aa40d211225cf34e914029a487eaa7ee58e05" exitCode=0
Sep 16 14:08:20.271066 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.270392    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal" event={"ID":"50a8e81bafdf533f0380af356a5122a5","Type":"ContainerDied","Data":"34a5ae4bc2ca62181584ed1fef8aa40d211225cf34e914029a487eaa7ee58e05"}
Sep 16 14:08:20.273647 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.273577    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"d677a63a75bd48b85e6e563530dcf363ad323c4b9b4e41688206b509af7dcfc5"}
Sep 16 14:08:20.275032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.275011    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="001f3b433d82a25a360fb552d372de586cc1b1aa4c9696df1aacea0870ac1fbb" exitCode=0
Sep 16 14:08:20.275149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.275065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"001f3b433d82a25a360fb552d372de586cc1b1aa4c9696df1aacea0870ac1fbb"}
Sep 16 14:08:20.276924 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.276888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8" event={"ID":"336f426e-189a-4e10-a4cc-05fcc62e2a58","Type":"ContainerStarted","Data":"1685d54a187ca8506e14d74cf3f3f11af55353675e1752e29b5884186e0f0dff"}
Sep 16 14:08:20.278486 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.278454    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7nplm" event={"ID":"8a942c87-447f-4ff1-9420-df5cc9e4318f","Type":"ContainerStarted","Data":"0c95f8d005f4effb676bdb144a6afe094ed43154d736872547cef9dcd9c7b72f"}
Sep 16 14:08:20.305954 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.305916    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7nplm" podStartSLOduration=3.046938658 podStartE2EDuration="21.30590268s" totalImagesPullingTime="18.258964022s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:20.305092289 +0000 UTC m=+21.711672513" watchObservedRunningTime="2026-09-16 14:08:20.30590268 +0000 UTC m=+21.712482888"
Sep 16 14:08:20.338178 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.338158    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:20.338646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:20.338626    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:21.177854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.177823    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:21.178014 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:21.177922    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:21.283021 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.282948    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8" event={"ID":"336f426e-189a-4e10-a4cc-05fcc62e2a58","Type":"ContainerStarted","Data":"44397b4f3c8448216ef653c2d9625bcbbd292e8088aaff8b1855b745b73ca193"}
Sep 16 14:08:21.284780 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.284754    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal" event={"ID":"50a8e81bafdf533f0380af356a5122a5","Type":"ContainerStarted","Data":"1b63ce2a25d8666c271a178ee95709c25e685006cfc75181629bee5d82e8c239"}
Sep 16 14:08:21.285097 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.285072    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:21.285431 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.285416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-zgc2b"
Sep 16 14:08:21.302388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.302350    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-sk9m8" podStartSLOduration=9.780509451 podStartE2EDuration="22.302335588s" totalImagesPullingTime="12.521826137s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:21.302017037 +0000 UTC m=+22.708597245" watchObservedRunningTime="2026-09-16 14:08:21.302335588 +0000 UTC m=+22.708915796"
Sep 16 14:08:21.317052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:21.317010    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-80.ec2.internal" podStartSLOduration=22.31699725 podStartE2EDuration="22.31699725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:21.316901459 +0000 UTC m=+22.723481668" watchObservedRunningTime="2026-09-16 14:08:21.31699725 +0000 UTC m=+22.723577458"
Sep 16 14:08:22.178125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:22.178090    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:22.178290 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:22.178104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:22.178290 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:22.178203    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:22.178290 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:22.178265    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:22.290252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:22.290220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"3c3ceca586283cf0814650da8611d812b528060004a2b6d783d8dbde299d1da5"}
Sep 16 14:08:23.182105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:23.182079    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:23.182349 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:23.182184    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:24.178904 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.178671    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:24.179333 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.178697    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:24.179333 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:24.179027    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:24.179333 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:24.179081    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:24.299665 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.299629    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" event={"ID":"79fa1313-3cd2-4310-a282-501928420a95","Type":"ContainerStarted","Data":"b46a0f7c57159cd5c2692ccd596baa154e9ed015f660a390fa51906bb0c1ac04"}
Sep 16 14:08:24.299958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.299936    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:24.316893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.316868    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:24.384775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:24.384716    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" podStartSLOduration=6.934588916 podStartE2EDuration="25.384695913s" totalImagesPullingTime="18.450106997s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:24.383195003 +0000 UTC m=+25.789775215" watchObservedRunningTime="2026-09-16 14:08:24.384695913 +0000 UTC m=+25.791276122"
Sep 16 14:08:25.178435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.178400    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:25.178642 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:25.178548    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:25.302888 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.302851    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 14:08:25.304128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.304099    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:25.322288 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.322253    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:25.571315 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.571029    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-klv79"]
Sep 16 14:08:25.571315 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.571178    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:25.571315 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:25.571305    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:25.574803 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.574776    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtwhx"]
Sep 16 14:08:25.575077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.574895    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:25.575077 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:25.574996    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:25.575511 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.575488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xx49p"]
Sep 16 14:08:25.575634 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:25.575619    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:25.575727 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:25.575708    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:26.305113 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:26.305077    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 14:08:27.178514 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.178487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:27.178692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.178487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:27.178692 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:27.178597    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:27.178692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.178487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:27.178854 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:27.178689    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:27.178854 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:27.178758    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:27.309041 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.309006    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="b8ca719ecd565d8c29f5fe2c33b08048d30913ac3b48de9e38afa5d36fed787d" exitCode=0
Sep 16 14:08:27.309447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.309060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"b8ca719ecd565d8c29f5fe2c33b08048d30913ac3b48de9e38afa5d36fed787d"}
Sep 16 14:08:27.309447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:27.309263    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 14:08:29.179387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.179162    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:29.179818 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.179217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:29.179818 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:29.179476    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:29.179818 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.179242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:29.179818 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:29.179533    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:29.179818 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:29.179645    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:29.316303 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.316270    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="c29ee4775131380c2292df49fbbb3640e4c322c80798a388e4a286722bdaca88" exitCode=0
Sep 16 14:08:29.316451 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.316329    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"c29ee4775131380c2292df49fbbb3640e4c322c80798a388e4a286722bdaca88"}
Sep 16 14:08:29.763982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.763948    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:08:29.764205 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.764190    2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 16 14:08:29.775449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.775392    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" podUID="79fa1313-3cd2-4310-a282-501928420a95" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 16 14:08:29.784821 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:29.784788    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg" podUID="79fa1313-3cd2-4310-a282-501928420a95" containerName="ovnkube-controller" probeResult="failure" output=""
Sep 16 14:08:30.321264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:30.321226    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="5835d260d6b4b647973d4b706588ac044248139f0936f55b29da8a48809f81ce" exitCode=0
Sep 16 14:08:30.321598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:30.321285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"5835d260d6b4b647973d4b706588ac044248139f0936f55b29da8a48809f81ce"}
Sep 16 14:08:31.178781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.178749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:31.178977 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.178749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:31.178977 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.178903    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-klv79" podUID="ddcf2bf3-dad0-47a2-a1b1-a852918c4591"
Sep 16 14:08:31.178977 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.178749    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:31.178977 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.178957    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-xx49p" podUID="18a21a6e-555b-453b-a36f-db4d3aad13d9"
Sep 16 14:08:31.179179 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.179048    2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-xtwhx" podUID="85aa1096-f9c0-4677-a9a4-e86c23d4c4ea"
Sep 16 14:08:31.793313 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.793280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:31.793691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.793324    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:31.793691 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.793426    2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:31.793691 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.793463    2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:31.793691 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.793488    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs podName:85aa1096-f9c0-4677-a9a4-e86c23d4c4ea nodeName:}" failed. No retries permitted until 2026-09-16 14:09:03.793472215 +0000 UTC m=+65.200052401 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs") pod "network-metrics-daemon-xtwhx" (UID: "85aa1096-f9c0-4677-a9a4-e86c23d4c4ea") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 16 14:08:31.793691 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.793503    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret podName:ddcf2bf3-dad0-47a2-a1b1-a852918c4591 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:03.793496474 +0000 UTC m=+65.200076660 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret") pod "global-pull-secret-syncer-klv79" (UID: "ddcf2bf3-dad0-47a2-a1b1-a852918c4591") : object "kube-system"/"original-pull-secret" not registered
Sep 16 14:08:31.894456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:31.894378    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:31.894616 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.894582    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 16 14:08:31.894616 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.894606    2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 16 14:08:31.894697 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.894619    2580 projected.go:196] Error preparing data for projected volume kube-api-access-nz78j for pod openshift-network-diagnostics/network-check-target-xx49p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:31.894697 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:31.894686    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j podName:18a21a6e-555b-453b-a36f-db4d3aad13d9 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:03.894666725 +0000 UTC m=+65.301246911 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-nz78j" (UniqueName: "kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j") pod "network-check-target-xx49p" (UID: "18a21a6e-555b-453b-a36f-db4d3aad13d9") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 16 14:08:32.339883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.339849    2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-80.ec2.internal" event="NodeReady"
Sep 16 14:08:32.340054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.339995    2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 16 14:08:32.402788 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.402438    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:08:32.404476 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.404455    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.406198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.406175    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-97pq7"]
Sep 16 14:08:32.407119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.407093    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 16 14:08:32.407801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.407623    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 16 14:08:32.407801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.407769    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 16 14:08:32.407960 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.407772    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-ffjnt\""
Sep 16 14:08:32.410914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.410654    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-njkrv"]
Sep 16 14:08:32.411103 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.410934    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.413549 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.413530    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.414511 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.414482    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 16 14:08:32.414633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.414483    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 16 14:08:32.418838 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.418818    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 16 14:08:32.419130 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.419105    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 16 14:08:32.419130 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.419110    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8lck4\""
Sep 16 14:08:32.419271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.419109    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 16 14:08:32.419271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.419256    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8lnxj\""
Sep 16 14:08:32.419374 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.419346    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 16 14:08:32.438692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.438641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-njkrv"]
Sep 16 14:08:32.447467 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.447441    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:08:32.484237 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.484212    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-97pq7"]
Sep 16 14:08:32.500066 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500043    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500107    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500133    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeebefad-ae12-4cb5-9cbf-516972fe339e-config-volume\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.500264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500153    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.500330 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500314    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500413 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500369    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500413 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500442    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500476    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeebefad-ae12-4cb5-9cbf-516972fe339e-tmp-dir\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.500587 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500497    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25nft\" (UniqueName: \"kubernetes.io/projected/aeebefad-ae12-4cb5-9cbf-516972fe339e-kube-api-access-25nft\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.500587 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500535    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aacf9cf-706a-401c-b3b1-73f57310a096-cert\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.500587 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500570    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cvdxt\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.500702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.500606    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jnc65\" (UniqueName: \"kubernetes.io/projected/9aacf9cf-706a-401c-b3b1-73f57310a096-kube-api-access-jnc65\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.601893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.601808    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aacf9cf-706a-401c-b3b1-73f57310a096-cert\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.601893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.601848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cvdxt\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.601893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.601874    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jnc65\" (UniqueName: \"kubernetes.io/projected/9aacf9cf-706a-401c-b3b1-73f57310a096-kube-api-access-jnc65\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.602165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.601917    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602075    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602107    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeebefad-ae12-4cb5-9cbf-516972fe339e-config-volume\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.602165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602130    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.602365 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:32.602209    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:32.602365 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:32.602262    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls podName:aeebefad-ae12-4cb5-9cbf-516972fe339e nodeName:}" failed. No retries permitted until 2026-09-16 14:08:33.102248504 +0000 UTC m=+34.508828690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls") pod "dns-default-njkrv" (UID: "aeebefad-ae12-4cb5-9cbf-516972fe339e") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:32.602474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602438    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602472    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602542    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.602603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602589    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeebefad-ae12-4cb5-9cbf-516972fe339e-tmp-dir\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.602795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.602613    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25nft\" (UniqueName: \"kubernetes.io/projected/aeebefad-ae12-4cb5-9cbf-516972fe339e-kube-api-access-25nft\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.603355 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:32.603026    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:08:32.603355 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:32.603046    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758dbd5bcc-4rmfk: secret "image-registry-tls" not found
Sep 16 14:08:32.603355 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:32.603121    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls podName:7e68ab3c-d6b6-4498-a266-f390ed6ca2de nodeName:}" failed. No retries permitted until 2026-09-16 14:08:33.103105298 +0000 UTC m=+34.509685484 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls") pod "image-registry-758dbd5bcc-4rmfk" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de") : secret "image-registry-tls" not found
Sep 16 14:08:32.603355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.603227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/aeebefad-ae12-4cb5-9cbf-516972fe339e-config-volume\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.603355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.603315    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.603659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.603372    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/aeebefad-ae12-4cb5-9cbf-516972fe339e-tmp-dir\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.603659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.603380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.604052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.604021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.606955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.606822    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.607054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.606863    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.607054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.606876    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9aacf9cf-706a-401c-b3b1-73f57310a096-cert\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.611287 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.611262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cvdxt\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.614148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.614127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:32.614256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.614237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25nft\" (UniqueName: \"kubernetes.io/projected/aeebefad-ae12-4cb5-9cbf-516972fe339e-kube-api-access-25nft\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:32.614334 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.614317    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jnc65\" (UniqueName: \"kubernetes.io/projected/9aacf9cf-706a-401c-b3b1-73f57310a096-kube-api-access-jnc65\") pod \"ingress-canary-97pq7\" (UID: \"9aacf9cf-706a-401c-b3b1-73f57310a096\") " pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:32.727066 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:32.727033    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-97pq7"
Sep 16 14:08:33.010467 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.010429    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-97pq7"]
Sep 16 14:08:33.015441 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:33.015407    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aacf9cf_706a_401c_b3b1_73f57310a096.slice/crio-e9b125546376943451d7bb12818eacd2bc799900904ca267ae6a1db396d4eadf WatchSource:0}: Error finding container e9b125546376943451d7bb12818eacd2bc799900904ca267ae6a1db396d4eadf: Status 404 returned error can't find the container with id e9b125546376943451d7bb12818eacd2bc799900904ca267ae6a1db396d4eadf
Sep 16 14:08:33.107376 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.107340    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:33.107376 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.107386    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:33.107603 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:33.107498    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:33.107603 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:33.107581    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls podName:aeebefad-ae12-4cb5-9cbf-516972fe339e nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.107551189 +0000 UTC m=+35.514131380 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls") pod "dns-default-njkrv" (UID: "aeebefad-ae12-4cb5-9cbf-516972fe339e") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:33.107603 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:33.107502    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:08:33.107730 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:33.107607    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758dbd5bcc-4rmfk: secret "image-registry-tls" not found
Sep 16 14:08:33.107730 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:33.107648    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls podName:7e68ab3c-d6b6-4498-a266-f390ed6ca2de nodeName:}" failed. No retries permitted until 2026-09-16 14:08:34.107634829 +0000 UTC m=+35.514215030 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls") pod "image-registry-758dbd5bcc-4rmfk" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de") : secret "image-registry-tls" not found
Sep 16 14:08:33.177874 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.177841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:08:33.178047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.177841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:08:33.178280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.177841    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:08:33.180589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.180415    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:08:33.180589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.180448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:08:33.180589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.180459    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:08:33.180589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.180491    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4gh8\""
Sep 16 14:08:33.180850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.180775    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mpwg\""
Sep 16 14:08:33.181211 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.181019    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:08:33.330322 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:33.330239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-97pq7" event={"ID":"9aacf9cf-706a-401c-b3b1-73f57310a096","Type":"ContainerStarted","Data":"e9b125546376943451d7bb12818eacd2bc799900904ca267ae6a1db396d4eadf"}
Sep 16 14:08:34.115457 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:34.115421    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:34.115469    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:34.115603    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:34.115611    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:34.115697    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls podName:aeebefad-ae12-4cb5-9cbf-516972fe339e nodeName:}" failed. No retries permitted until 2026-09-16 14:08:36.115666304 +0000 UTC m=+37.522246503 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls") pod "dns-default-njkrv" (UID: "aeebefad-ae12-4cb5-9cbf-516972fe339e") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:34.115615    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758dbd5bcc-4rmfk: secret "image-registry-tls" not found
Sep 16 14:08:34.115983 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:34.115757    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls podName:7e68ab3c-d6b6-4498-a266-f390ed6ca2de nodeName:}" failed. No retries permitted until 2026-09-16 14:08:36.115743304 +0000 UTC m=+37.522323515 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls") pod "image-registry-758dbd5bcc-4rmfk" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de") : secret "image-registry-tls" not found
Sep 16 14:08:35.338606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:35.338553    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="18ddd3a7b9daecd4b58897dbe2010425a59e54fba0db2e9abed6bdb5b3d72fae" exitCode=0
Sep 16 14:08:35.338606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:35.338595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"18ddd3a7b9daecd4b58897dbe2010425a59e54fba0db2e9abed6bdb5b3d72fae"}
Sep 16 14:08:36.133155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.133116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:36.133340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.133175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:36.133340 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:36.133256    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:36.133340 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:36.133285    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:08:36.133340 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:36.133298    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758dbd5bcc-4rmfk: secret "image-registry-tls" not found
Sep 16 14:08:36.133340 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:36.133327    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls podName:aeebefad-ae12-4cb5-9cbf-516972fe339e nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.133311016 +0000 UTC m=+41.539891203 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls") pod "dns-default-njkrv" (UID: "aeebefad-ae12-4cb5-9cbf-516972fe339e") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:36.133672 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:36.133348    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls podName:7e68ab3c-d6b6-4498-a266-f390ed6ca2de nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.133337605 +0000 UTC m=+41.539917791 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls") pod "image-registry-758dbd5bcc-4rmfk" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de") : secret "image-registry-tls" not found
Sep 16 14:08:36.345394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.345211    2580 generic.go:356] "Generic (PLEG): container finished" podID="bee02a73-e799-45f1-88c1-91c30f3428c0" containerID="fbfb1232b15af6a2c08b30af9f815348f573af47fa6c1d57ee161d576d4f1c9d" exitCode=0
Sep 16 14:08:36.345394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.345302    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerDied","Data":"fbfb1232b15af6a2c08b30af9f815348f573af47fa6c1d57ee161d576d4f1c9d"}
Sep 16 14:08:36.755447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.755415    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-2plzs"]
Sep 16 14:08:36.790009 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.789985    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2plzs"]
Sep 16 14:08:36.790141 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.790099    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:36.793908 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.793881    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 16 14:08:36.794058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.793941    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-c7jvw\""
Sep 16 14:08:36.794058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.793884    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 16 14:08:36.794453 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.794436    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 16 14:08:36.794585 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.794526    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 16 14:08:36.939142 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.939101    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:36.939312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.939174    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0479fe57-ca07-4f5c-9059-981c5ffa9f79-crio-socket\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:36.939312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.939199    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:36.939312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.939214    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-api-access-d6qlf\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:36.939312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:36.939238    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0479fe57-ca07-4f5c-9059-981c5ffa9f79-data-volume\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040164 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040201    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0479fe57-ca07-4f5c-9059-981c5ffa9f79-crio-socket\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040255    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-api-access-d6qlf\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040434 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0479fe57-ca07-4f5c-9059-981c5ffa9f79-data-volume\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040434 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.040299    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.040434 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.040366    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.540345624 +0000 UTC m=+38.946925831 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.040600 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.040456    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.040600 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040525    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/0479fe57-ca07-4f5c-9059-981c5ffa9f79-crio-socket\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.040600 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.040568    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:37.540537655 +0000 UTC m=+38.947117852 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.040740 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.040618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/0479fe57-ca07-4f5c-9059-981c5ffa9f79-data-volume\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.062627 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.062599    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d6qlf\" (UniqueName: \"kubernetes.io/projected/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-api-access-d6qlf\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.350347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.350312    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-97pq7" event={"ID":"9aacf9cf-706a-401c-b3b1-73f57310a096","Type":"ContainerStarted","Data":"270c4fb91cc0930f85011a19da768b2025e425c074f335d0287272ce29272a1a"}
Sep 16 14:08:37.353927 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.353905    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" event={"ID":"bee02a73-e799-45f1-88c1-91c30f3428c0","Type":"ContainerStarted","Data":"68d197ad45f3d524bd92f533453f5df95fc4b2ec73a6bf3b48a0ff4ac1bcdd34"}
Sep 16 14:08:37.368841 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.368800    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-97pq7" podStartSLOduration=1.226496645 podStartE2EDuration="5.368786674s" totalImagesPullingTime="4.142290029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:37.368420125 +0000 UTC m=+38.775000332" watchObservedRunningTime="2026-09-16 14:08:37.368786674 +0000 UTC m=+38.775366881"
Sep 16 14:08:37.391528 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.391489    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mh2p9" podStartSLOduration=7.274380672 podStartE2EDuration="38.391479208s" totalImagesPullingTime="31.117098536s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:37.391413245 +0000 UTC m=+38.797993464" watchObservedRunningTime="2026-09-16 14:08:37.391479208 +0000 UTC m=+38.798059395"
Sep 16 14:08:37.545026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.544946    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.545026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:37.544999    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:37.545218 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.545077    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:37.545218 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.545080    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.545218 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.545144    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:38.545127489 +0000 UTC m=+39.951707675 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:37.545218 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:37.545159    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:38.545152221 +0000 UTC m=+39.951732407 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:38.553514 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:38.553474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:38.553944 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:38.553536    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:38.553944 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:38.553622    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:38.553944 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:38.553687    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.553672689 +0000 UTC m=+41.960252893 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:38.553944 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:38.553625    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:38.553944 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:38.553768    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:40.553755997 +0000 UTC m=+41.960336183 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:38.709611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:38.709576    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kz8kh_a1c0520e-773d-4d35-8f4a-394f677fea8d/dns-node-resolver/0.log"
Sep 16 14:08:39.909815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:39.909789    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgpt4_17ba2d77-eb74-4d56-a93f-7df4d41b2ac5/node-ca/0.log"
Sep 16 14:08:40.166443 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:40.166353    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:40.166443 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:40.166404    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:40.166684 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.166513    2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 16 14:08:40.166684 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.166541    2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 16 14:08:40.166684 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.166574    2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-758dbd5bcc-4rmfk: secret "image-registry-tls" not found
Sep 16 14:08:40.166684 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.166587    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls podName:aeebefad-ae12-4cb5-9cbf-516972fe339e nodeName:}" failed. No retries permitted until 2026-09-16 14:08:48.166569828 +0000 UTC m=+49.573150026 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls") pod "dns-default-njkrv" (UID: "aeebefad-ae12-4cb5-9cbf-516972fe339e") : secret "dns-default-metrics-tls" not found
Sep 16 14:08:40.166684 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.166621    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls podName:7e68ab3c-d6b6-4498-a266-f390ed6ca2de nodeName:}" failed. No retries permitted until 2026-09-16 14:08:48.166606318 +0000 UTC m=+49.573186504 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls") pod "image-registry-758dbd5bcc-4rmfk" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de") : secret "image-registry-tls" not found
Sep 16 14:08:40.511255 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:40.511229    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-97pq7_9aacf9cf-706a-401c-b3b1-73f57310a096/serve-healthcheck-canary/0.log"
Sep 16 14:08:40.569669 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:40.569642    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:40.569821 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:40.569691    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:40.569821 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.569765    2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 16 14:08:40.569821 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.569799    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:40.569937 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.569839    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:44.569821187 +0000 UTC m=+45.976401392 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : configmap "kube-rbac-proxy" not found
Sep 16 14:08:40.569937 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:40.569852    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:44.569846517 +0000 UTC m=+45.976426702 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:44.604286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:44.604243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:44.604728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:44.604309    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:44.604728 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:44.604395    2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:44.604728 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:08:44.604464    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls podName:0479fe57-ca07-4f5c-9059-981c5ffa9f79 nodeName:}" failed. No retries permitted until 2026-09-16 14:08:52.604447322 +0000 UTC m=+54.011027534 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls") pod "insights-runtime-extractor-2plzs" (UID: "0479fe57-ca07-4f5c-9059-981c5ffa9f79") : secret "insights-runtime-extractor-tls" not found
Sep 16 14:08:44.604875 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:44.604854    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/0479fe57-ca07-4f5c-9059-981c5ffa9f79-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:48.234024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.233988    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:48.234399 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.234033    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:48.236297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.236275    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/aeebefad-ae12-4cb5-9cbf-516972fe339e-metrics-tls\") pod \"dns-default-njkrv\" (UID: \"aeebefad-ae12-4cb5-9cbf-516972fe339e\") " pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:48.236422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.236352    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"image-registry-758dbd5bcc-4rmfk\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") " pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:48.318450 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.318412    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:48.332559 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.332527    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:48.630150 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.630069    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:08:48.633799 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:48.633775    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e68ab3c_d6b6_4498_a266_f390ed6ca2de.slice/crio-0c81947853b43d15304d7211d6ec3cc37873bc6107eb82dbdff747298c977a4f WatchSource:0}: Error finding container 0c81947853b43d15304d7211d6ec3cc37873bc6107eb82dbdff747298c977a4f: Status 404 returned error can't find the container with id 0c81947853b43d15304d7211d6ec3cc37873bc6107eb82dbdff747298c977a4f
Sep 16 14:08:48.658598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:48.658576    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-njkrv"]
Sep 16 14:08:48.661339 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:48.661311    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaeebefad_ae12_4cb5_9cbf_516972fe339e.slice/crio-d4ed9dceec8c65cb303e484ff9c98bf8eaced78b26cdcb836e136dd5b5afafe5 WatchSource:0}: Error finding container d4ed9dceec8c65cb303e484ff9c98bf8eaced78b26cdcb836e136dd5b5afafe5: Status 404 returned error can't find the container with id d4ed9dceec8c65cb303e484ff9c98bf8eaced78b26cdcb836e136dd5b5afafe5
Sep 16 14:08:49.383199 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:49.383154    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-njkrv" event={"ID":"aeebefad-ae12-4cb5-9cbf-516972fe339e","Type":"ContainerStarted","Data":"d4ed9dceec8c65cb303e484ff9c98bf8eaced78b26cdcb836e136dd5b5afafe5"}
Sep 16 14:08:49.385011 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:49.384982    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" event={"ID":"7e68ab3c-d6b6-4498-a266-f390ed6ca2de","Type":"ContainerStarted","Data":"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"}
Sep 16 14:08:49.385178 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:49.385030    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" event={"ID":"7e68ab3c-d6b6-4498-a266-f390ed6ca2de","Type":"ContainerStarted","Data":"0c81947853b43d15304d7211d6ec3cc37873bc6107eb82dbdff747298c977a4f"}
Sep 16 14:08:49.385178 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:49.385069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:08:49.404996 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:49.404940    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" podStartSLOduration=27.404919594 podStartE2EDuration="27.404919594s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:49.403896006 +0000 UTC m=+50.810476214" watchObservedRunningTime="2026-09-16 14:08:49.404919594 +0000 UTC m=+50.811499805"
Sep 16 14:08:51.391888 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:51.391850    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-njkrv" event={"ID":"aeebefad-ae12-4cb5-9cbf-516972fe339e","Type":"ContainerStarted","Data":"61d7fdb4f990a60cece097c99d977baaeccff51ae3d872328b8a7dfc10c77392"}
Sep 16 14:08:51.392365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:51.391902    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-njkrv" event={"ID":"aeebefad-ae12-4cb5-9cbf-516972fe339e","Type":"ContainerStarted","Data":"fa65ee7234903d147601533be7dced2681ffdc790e6bea24f920476acb26b043"}
Sep 16 14:08:51.392365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:51.391973    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-njkrv"
Sep 16 14:08:52.667292 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:52.667240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:52.669450 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:52.669431    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/0479fe57-ca07-4f5c-9059-981c5ffa9f79-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-2plzs\" (UID: \"0479fe57-ca07-4f5c-9059-981c5ffa9f79\") " pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:52.701693 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:52.701668    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-2plzs"
Sep 16 14:08:52.972235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:52.972184    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-njkrv" podStartSLOduration=19.22415392 podStartE2EDuration="20.972167069s" totalImagesPullingTime="1.748013149s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:51.422947641 +0000 UTC m=+52.829527848" watchObservedRunningTime="2026-09-16 14:08:52.972167069 +0000 UTC m=+54.378747267"
Sep 16 14:08:52.972391 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:52.972365    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-2plzs"]
Sep 16 14:08:52.975633 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:08:52.975611    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0479fe57_ca07_4f5c_9059_981c5ffa9f79.slice/crio-368fb3ae604d879152640e38b496056e1d9eb8dd0463c159aff64eef899eabd7 WatchSource:0}: Error finding container 368fb3ae604d879152640e38b496056e1d9eb8dd0463c159aff64eef899eabd7: Status 404 returned error can't find the container with id 368fb3ae604d879152640e38b496056e1d9eb8dd0463c159aff64eef899eabd7
Sep 16 14:08:53.398615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:53.398514    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2plzs" event={"ID":"0479fe57-ca07-4f5c-9059-981c5ffa9f79","Type":"ContainerStarted","Data":"41b0750f754a63cea78a833be596fded779bfecb54352882e4e8918983c9eb8e"}
Sep 16 14:08:53.398615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:53.398611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2plzs" event={"ID":"0479fe57-ca07-4f5c-9059-981c5ffa9f79","Type":"ContainerStarted","Data":"368fb3ae604d879152640e38b496056e1d9eb8dd0463c159aff64eef899eabd7"}
Sep 16 14:08:54.403500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:54.403467    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2plzs" event={"ID":"0479fe57-ca07-4f5c-9059-981c5ffa9f79","Type":"ContainerStarted","Data":"c6caa48b40cd0b1792152e79580f7f268dc04dcb14e8d9c3dc35abf603018138"}
Sep 16 14:08:56.413032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:56.412992    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-2plzs" event={"ID":"0479fe57-ca07-4f5c-9059-981c5ffa9f79","Type":"ContainerStarted","Data":"10fb21a29bf7b062234dd6025bc08d9e83e0f414e1e0c0770c56588accd09633"}
Sep 16 14:08:56.434463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:56.434413    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-2plzs" podStartSLOduration=17.974429253 podStartE2EDuration="20.434398591s" totalImagesPullingTime="2.459969338s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:08:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:08:56.433054016 +0000 UTC m=+57.839634237" watchObservedRunningTime="2026-09-16 14:08:56.434398591 +0000 UTC m=+57.840978799"
Sep 16 14:08:59.452185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.452155    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"]
Sep 16 14:08:59.455083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.455064    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.458416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.458390    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 16 14:08:59.458537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.458426    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 16 14:08:59.458694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.458678    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 16 14:08:59.458694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.458691    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 16 14:08:59.459395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.459379    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 16 14:08:59.459609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.459594    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 16 14:08:59.459849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.459835    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 16 14:08:59.479314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.479292    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"]
Sep 16 14:08:59.511716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.511798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.511798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511738    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.511798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511770    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.511901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511863    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-snckj\" (UniqueName: \"kubernetes.io/projected/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-kube-api-access-snckj\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.511901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.511893    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.555022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.554997    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:08:59.613079 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-snckj\" (UniqueName: \"kubernetes.io/projected/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-kube-api-access-snckj\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.613079 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613084    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.613274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.613274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613153    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.613274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613169    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.613274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.613197    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.614030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.614001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-additional-service-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.619059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.615816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.619059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.615948    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.619059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.616039    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-hub\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.619578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.619542    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-ca\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.622779 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.622760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-snckj\" (UniqueName: \"kubernetes.io/projected/9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8-kube-api-access-snckj\") pod \"cluster-proxy-proxy-agent-775dd75d85-mrs4t\" (UID: \"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.772718 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.772645    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"
Sep 16 14:08:59.785166 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:08:59.785144    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jgppg"
Sep 16 14:09:00.084153 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:00.084077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t"]
Sep 16 14:09:00.086980 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:00.086955    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bb157a4_c5c7_4d51_a6cb_7d684e28c6f8.slice/crio-c548cc2367382a51fa68b3f2370b2cf15fd2ee4e4bcda6f2dbcb61cd3881e52b WatchSource:0}: Error finding container c548cc2367382a51fa68b3f2370b2cf15fd2ee4e4bcda6f2dbcb61cd3881e52b: Status 404 returned error can't find the container with id c548cc2367382a51fa68b3f2370b2cf15fd2ee4e4bcda6f2dbcb61cd3881e52b
Sep 16 14:09:00.425914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:00.425826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t" event={"ID":"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8","Type":"ContainerStarted","Data":"c548cc2367382a51fa68b3f2370b2cf15fd2ee4e4bcda6f2dbcb61cd3881e52b"}
Sep 16 14:09:01.396724 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:01.396690    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-njkrv"
Sep 16 14:09:03.777545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.777515    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"]
Sep 16 14:09:03.779757 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.779743    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:03.781750 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.781722    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\""
Sep 16 14:09:03.782115 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.782098    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-crhcc\""
Sep 16 14:09:03.790714 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.790694    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"]
Sep 16 14:09:03.850124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.850096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-vqqhc\" (UID: \"d0e94b69-6556-4ab6-8f7f-2e3295b5e525\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:03.850210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.850165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:09:03.850248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.850209    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:09:03.852731 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.852708    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 16 14:09:03.852824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.852763    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 16 14:09:03.862350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.862331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/85aa1096-f9c0-4677-a9a4-e86c23d4c4ea-metrics-certs\") pod \"network-metrics-daemon-xtwhx\" (UID: \"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea\") " pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:09:03.862447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.862331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ddcf2bf3-dad0-47a2-a1b1-a852918c4591-original-pull-secret\") pod \"global-pull-secret-syncer-klv79\" (UID: \"ddcf2bf3-dad0-47a2-a1b1-a852918c4591\") " pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:09:03.950871 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.950844    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:09:03.950970 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.950903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-vqqhc\" (UID: \"d0e94b69-6556-4ab6-8f7f-2e3295b5e525\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:03.951034 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:03.951014    2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:09:03.951127 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:03.951114    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates podName:d0e94b69-6556-4ab6-8f7f-2e3295b5e525 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:04.451099184 +0000 UTC m=+65.857679383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates") pod "prometheus-operator-admission-webhook-5864cd99d7-vqqhc" (UID: "d0e94b69-6556-4ab6-8f7f-2e3295b5e525") : secret "prometheus-operator-admission-webhook-tls" not found
Sep 16 14:09:03.953435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.953418    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 16 14:09:03.963257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.963233    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 16 14:09:03.973667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:03.973639    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nz78j\" (UniqueName: \"kubernetes.io/projected/18a21a6e-555b-453b-a36f-db4d3aad13d9-kube-api-access-nz78j\") pod \"network-check-target-xx49p\" (UID: \"18a21a6e-555b-453b-a36f-db4d3aad13d9\") " pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:09:04.089383 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.089298    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-klv79"
Sep 16 14:09:04.098780 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.098760    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-x4gh8\""
Sep 16 14:09:04.104723 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.104705    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8mpwg\""
Sep 16 14:09:04.106445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.106432    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-xtwhx"
Sep 16 14:09:04.113321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.113302    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:09:04.363332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.363231    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-klv79"]
Sep 16 14:09:04.365518 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:04.365494    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddcf2bf3_dad0_47a2_a1b1_a852918c4591.slice/crio-1b7b692ef9c30e988f04dde864dd4a4273d6ce5c733527477e5352db798baf18 WatchSource:0}: Error finding container 1b7b692ef9c30e988f04dde864dd4a4273d6ce5c733527477e5352db798baf18: Status 404 returned error can't find the container with id 1b7b692ef9c30e988f04dde864dd4a4273d6ce5c733527477e5352db798baf18
Sep 16 14:09:04.395067 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.394073    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-xtwhx"]
Sep 16 14:09:04.398117 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:04.398091    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85aa1096_f9c0_4677_a9a4_e86c23d4c4ea.slice/crio-7712219a8a5762597a571739269f91fec4f702a3ec9cddc84f7364b9d36b057d WatchSource:0}: Error finding container 7712219a8a5762597a571739269f91fec4f702a3ec9cddc84f7364b9d36b057d: Status 404 returned error can't find the container with id 7712219a8a5762597a571739269f91fec4f702a3ec9cddc84f7364b9d36b057d
Sep 16 14:09:04.423573 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.423532    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-xx49p"]
Sep 16 14:09:04.426681 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:04.426658    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18a21a6e_555b_453b_a36f_db4d3aad13d9.slice/crio-60666d015915f36d872310e09df9df6008af62e4b95ac9decb22847901e5a205 WatchSource:0}: Error finding container 60666d015915f36d872310e09df9df6008af62e4b95ac9decb22847901e5a205: Status 404 returned error can't find the container with id 60666d015915f36d872310e09df9df6008af62e4b95ac9decb22847901e5a205
Sep 16 14:09:04.440709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.440683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtwhx" event={"ID":"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea","Type":"ContainerStarted","Data":"7712219a8a5762597a571739269f91fec4f702a3ec9cddc84f7364b9d36b057d"}
Sep 16 14:09:04.441841 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.441812    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-klv79" event={"ID":"ddcf2bf3-dad0-47a2-a1b1-a852918c4591","Type":"ContainerStarted","Data":"1b7b692ef9c30e988f04dde864dd4a4273d6ce5c733527477e5352db798baf18"}
Sep 16 14:09:04.443955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.443922    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t" event={"ID":"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8","Type":"ContainerStarted","Data":"5e57198d34a85fbda8ab15d6d8b0ba161d39c6e97062642623a006e2d1ab26dc"}
Sep 16 14:09:04.444047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.443958    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t" event={"ID":"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8","Type":"ContainerStarted","Data":"d830f08277c911a971cc9c126832104a5ed4a873b14443a345e92d189a12087a"}
Sep 16 14:09:04.444047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.443976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t" event={"ID":"9bb157a4-c5c7-4d51-a6cb-7d684e28c6f8","Type":"ContainerStarted","Data":"99d5b42ce4d23269344bd6d7f107e0594d6d836407eb0eb23567e113072fdabd"}
Sep 16 14:09:04.444981 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.444959    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xx49p" event={"ID":"18a21a6e-555b-453b-a36f-db4d3aad13d9","Type":"ContainerStarted","Data":"60666d015915f36d872310e09df9df6008af62e4b95ac9decb22847901e5a205"}
Sep 16 14:09:04.455282 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.455260    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-vqqhc\" (UID: \"d0e94b69-6556-4ab6-8f7f-2e3295b5e525\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:04.457754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.457736    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/d0e94b69-6556-4ab6-8f7f-2e3295b5e525-tls-certificates\") pod \"prometheus-operator-admission-webhook-5864cd99d7-vqqhc\" (UID: \"d0e94b69-6556-4ab6-8f7f-2e3295b5e525\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:04.465455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.465419    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-775dd75d85-mrs4t" podStartSLOduration=1.728877311 podStartE2EDuration="5.465407287s" totalImagesPullingTime="3.736529976s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:08:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:04.464591141 +0000 UTC m=+65.871171349" watchObservedRunningTime="2026-09-16 14:09:04.465407287 +0000 UTC m=+65.871987492"
Sep 16 14:09:04.688366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.688275    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:04.956093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:04.956065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"]
Sep 16 14:09:04.959629 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:04.959604    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0e94b69_6556_4ab6_8f7f_2e3295b5e525.slice/crio-c3400382bfef8115378dfdb5d103b27a5571de53102561ed59279ccbfa059a8e WatchSource:0}: Error finding container c3400382bfef8115378dfdb5d103b27a5571de53102561ed59279ccbfa059a8e: Status 404 returned error can't find the container with id c3400382bfef8115378dfdb5d103b27a5571de53102561ed59279ccbfa059a8e
Sep 16 14:09:05.450692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:05.450613    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc" event={"ID":"d0e94b69-6556-4ab6-8f7f-2e3295b5e525","Type":"ContainerStarted","Data":"c3400382bfef8115378dfdb5d103b27a5571de53102561ed59279ccbfa059a8e"}
Sep 16 14:09:06.457354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.457307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc" event={"ID":"d0e94b69-6556-4ab6-8f7f-2e3295b5e525","Type":"ContainerStarted","Data":"5e10c6440164775497e76683b4a39b57d81903c4ca1c99a148949307399739ac"}
Sep 16 14:09:06.457865 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.457499    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:06.460319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.460282    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtwhx" event={"ID":"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea","Type":"ContainerStarted","Data":"2d34b9d2c2f1924061f887c6069380d52acbf4f269b2c75d2aec27d27b0738b4"}
Sep 16 14:09:06.460447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.460338    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-xtwhx" event={"ID":"85aa1096-f9c0-4677-a9a4-e86c23d4c4ea","Type":"ContainerStarted","Data":"93e2385970cff6a84d1b2b2bcef7856c4739c7d897948453655f6c387bf0839d"}
Sep 16 14:09:06.463953 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.463933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc"
Sep 16 14:09:06.480884 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.480804    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-5864cd99d7-vqqhc" podStartSLOduration=2.211242652 podStartE2EDuration="3.48079096s" totalImagesPullingTime="1.269548308s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:06.479821582 +0000 UTC m=+67.886401790" watchObservedRunningTime="2026-09-16 14:09:06.48079096 +0000 UTC m=+67.887371169"
Sep 16 14:09:06.517392 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:06.517331    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-xtwhx" podStartSLOduration=66.160657193 podStartE2EDuration="1m7.517279812s" totalImagesPullingTime="1.356622619s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:06.516060881 +0000 UTC m=+67.922641090" watchObservedRunningTime="2026-09-16 14:09:06.517279812 +0000 UTC m=+67.923860021"
Sep 16 14:09:08.906663 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.906619    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:09:08.923433 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.923398    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.931777 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.931752    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 16 14:09:08.932329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.931940    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-w628n\""
Sep 16 14:09:08.932329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.932255    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 16 14:09:08.932765 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.932745    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 16 14:09:08.932897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.932778    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 16 14:09:08.932897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.932780    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 16 14:09:08.932996 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.932920    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 16 14:09:08.933043 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.933009    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 16 14:09:08.934101 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.934077    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:09:08.940148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.940126    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 16 14:09:08.988901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.988873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.988910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.988955    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.989029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.989055    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.989128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mwct\" (UniqueName: \"kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:08.989246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:08.989146    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090210    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mwct\" (UniqueName: \"kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090244    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090313    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090337    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.090711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.090624    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.091198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.091167    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.091198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.091189    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.091379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.091304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.091379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.091304    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.093138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.093115    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.093347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.093331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.099218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.099195    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mwct\" (UniqueName: \"kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct\") pod \"console-76f5dd85db-crxzz\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") " pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.235212 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.235172    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:09.559980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:09.559917    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:09:10.202661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.202626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:09:10.208897 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:10.208871    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fba751b_173d_4b6b_8d09_c4d57e7fa045.slice/crio-3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb WatchSource:0}: Error finding container 3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb: Status 404 returned error can't find the container with id 3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb
Sep 16 14:09:10.476766 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.476688    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-klv79" event={"ID":"ddcf2bf3-dad0-47a2-a1b1-a852918c4591","Type":"ContainerStarted","Data":"b30466cd17f356b977927dcbef33c4f35dcbf070ae432cee1c9fe700d415cee2"}
Sep 16 14:09:10.477846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.477807    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5dd85db-crxzz" event={"ID":"4fba751b-173d-4b6b-8d09-c4d57e7fa045","Type":"ContainerStarted","Data":"3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb"}
Sep 16 14:09:10.479109 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.479089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-xx49p" event={"ID":"18a21a6e-555b-453b-a36f-db4d3aad13d9","Type":"ContainerStarted","Data":"4bf7403177417e080349ec4ecaaa895b6c6cb3b613794367df3b64c08f58f4b3"}
Sep 16 14:09:10.479222 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.479201    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:09:10.505961 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:10.505914    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-klv79" podStartSLOduration=65.92465612 podStartE2EDuration="1m11.50590268s" totalImagesPullingTime="5.58124656s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:10.504261785 +0000 UTC m=+71.910842222" watchObservedRunningTime="2026-09-16 14:09:10.50590268 +0000 UTC m=+71.912482888"
Sep 16 14:09:13.230624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.230570    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-xx49p" podStartSLOduration=68.72567042 podStartE2EDuration="1m14.230539537s" totalImagesPullingTime="5.504869117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:07:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:10.565306326 +0000 UTC m=+71.971886535" watchObservedRunningTime="2026-09-16 14:09:13.230539537 +0000 UTC m=+74.637119744"
Sep 16 14:09:13.231029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.230933    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-588tg"]
Sep 16 14:09:13.233063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.233048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.235439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235409    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 16 14:09:13.235593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235460    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 16 14:09:13.235593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235515    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 16 14:09:13.235593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235529    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 16 14:09:13.235593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235530    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 16 14:09:13.235785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.235766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 16 14:09:13.236146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.236132    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-g65fn\""
Sep 16 14:09:13.322717 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322687    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-wtmp\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.322896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.322896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.322896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322777    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-metrics-client-ca\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.322896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sdm5m\" (UniqueName: \"kubernetes.io/projected/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-kube-api-access-sdm5m\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.323029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322899    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.323029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322922    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-root\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.323029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322952    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-sys\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.323029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.322971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-textfile\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423479 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-sys\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423479 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423476    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-textfile\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-wtmp\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423526    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423549    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-sys\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423578    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423623    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-metrics-client-ca\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sdm5m\" (UniqueName: \"kubernetes.io/projected/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-kube-api-access-sdm5m\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-wtmp\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.423725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.424062 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:13.423782    2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
Sep 16 14:09:13.424062 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:13.423867    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls podName:4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b nodeName:}" failed. No retries permitted until 2026-09-16 14:09:13.923847782 +0000 UTC m=+75.330427983 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls") pod "node-exporter-588tg" (UID: "4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b") : secret "node-exporter-tls" not found
Sep 16 14:09:13.424062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423885    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-root\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.424062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423899    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-textfile\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.424062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.423949    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-root\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.424273 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.424235    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-metrics-client-ca\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.424273 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.424237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-accelerators-collector-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.425854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.425831    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.433226 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.433204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sdm5m\" (UniqueName: \"kubernetes.io/projected/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-kube-api-access-sdm5m\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.491913 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.491837    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5dd85db-crxzz" event={"ID":"4fba751b-173d-4b6b-8d09-c4d57e7fa045","Type":"ContainerStarted","Data":"40473dc0d87e35c108ddbb19faf54a4e2854b64e52b175845679e72ed8792db6"}
Sep 16 14:09:13.508686 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.508637    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-76f5dd85db-crxzz" podStartSLOduration=2.876185231 podStartE2EDuration="5.50862056s" totalImagesPullingTime="2.632435329s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:13.508402978 +0000 UTC m=+74.914983189" watchObservedRunningTime="2026-09-16 14:09:13.50862056 +0000 UTC m=+74.915200768"
Sep 16 14:09:13.928015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.927931    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:13.930286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:13.930251    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b-node-exporter-tls\") pod \"node-exporter-588tg\" (UID: \"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b\") " pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:14.142731 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.142683    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-588tg"
Sep 16 14:09:14.153419 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:14.153387    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4128f4e8_c6fa_4e48_9dc1_cbb2a05c6b3b.slice/crio-739d472b46ef1f719858ad8f6ff1480e9f6dac53ef2ec6702965eda17f7551da WatchSource:0}: Error finding container 739d472b46ef1f719858ad8f6ff1480e9f6dac53ef2ec6702965eda17f7551da: Status 404 returned error can't find the container with id 739d472b46ef1f719858ad8f6ff1480e9f6dac53ef2ec6702965eda17f7551da
Sep 16 14:09:14.285689 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.285656    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:09:14.288678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.288651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.291379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.291357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 14:09:14.291606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.291590    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 14:09:14.291696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.291679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 14:09:14.291749 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.291697    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 14:09:14.291950 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.291934    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 14:09:14.292347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.292329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gbwfg\""
Sep 16 14:09:14.292459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.292349    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 14:09:14.292459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.292355    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 14:09:14.292459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.292357    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 14:09:14.292459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.292329    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 14:09:14.306142 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.306117    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:09:14.331743 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331712    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.331886 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331754    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.331886 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331771    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.331886 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331790    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvkdh\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.331886 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331911    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331929    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331950    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331971    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.331988    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.332040    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.332071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.332197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.332091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.432968 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.432933    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvkdh\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.432968 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.432969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.432993    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433397 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:14.433320    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle podName:be394e4f-bf76-43c3-8d4a-d2ec54563483 nodeName:}" failed. No retries permitted until 2026-09-16 14:09:14.933296257 +0000 UTC m=+76.339876458 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483") : configmap references non-existent config key: ca-bundle.crt
Sep 16 14:09:14.433397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433350    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433443    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.433691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.433474    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.434544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.434519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.434967 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.434818    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.436088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.436041    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.436196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.436087    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.436333 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.436311    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.436656 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.436632    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.436761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.436693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.437146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.437118    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.437405 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.437389    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.437495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.437480    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.438428 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.438408    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.444364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.444347    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvkdh\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.497022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.496983    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-588tg" event={"ID":"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b","Type":"ContainerStarted","Data":"739d472b46ef1f719858ad8f6ff1480e9f6dac53ef2ec6702965eda17f7551da"}
Sep 16 14:09:14.938285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.938249    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:14.939039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:14.939014    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:15.201117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:15.201081    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:09:15.501986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:15.501902    2580 generic.go:356] "Generic (PLEG): container finished" podID="4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b" containerID="47928f7af1bc1148d55138bd39cbcc9f5bf8e8644e7593d4fe2b6a8e5b76d120" exitCode=0
Sep 16 14:09:15.501986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:15.501964    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-588tg" event={"ID":"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b","Type":"ContainerDied","Data":"47928f7af1bc1148d55138bd39cbcc9f5bf8e8644e7593d4fe2b6a8e5b76d120"}
Sep 16 14:09:15.556429 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:15.556403    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:09:15.559709 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:15.559675    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe394e4f_bf76_43c3_8d4a_d2ec54563483.slice/crio-cd9be160d1594970239d35b4636cccbe323ea410d560f4c9191b9278c920d75a WatchSource:0}: Error finding container cd9be160d1594970239d35b4636cccbe323ea410d560f4c9191b9278c920d75a: Status 404 returned error can't find the container with id cd9be160d1594970239d35b4636cccbe323ea410d560f4c9191b9278c920d75a
Sep 16 14:09:16.507876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:16.507839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-588tg" event={"ID":"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b","Type":"ContainerStarted","Data":"a4f538685f1f104d53d7d870464e8ca5619c8c37ffaeeefd9664a28f6949436f"}
Sep 16 14:09:16.508327 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:16.507901    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-588tg" event={"ID":"4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b","Type":"ContainerStarted","Data":"351d042bd3946998f3cb2c757c45bfe28e1cdb6df92c9e4ad4bc174c1a44bff6"}
Sep 16 14:09:16.509090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:16.509071    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"cd9be160d1594970239d35b4636cccbe323ea410d560f4c9191b9278c920d75a"}
Sep 16 14:09:16.528970 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:16.528925    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-588tg" podStartSLOduration=2.323267939 podStartE2EDuration="3.5289121s" totalImagesPullingTime="1.142260738s" totalInitContainerRuntime="63.383423ms" isStatefulPod=true podCreationTimestamp="2026-09-16 14:09:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:16.528766373 +0000 UTC m=+77.935346580" watchObservedRunningTime="2026-09-16 14:09:16.5289121 +0000 UTC m=+77.935492307"
Sep 16 14:09:17.687913 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.687880    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6488dffc95-pksr7"]
Sep 16 14:09:17.689795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.689779    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.693548 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693518    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
Sep 16 14:09:17.693548 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693526    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
Sep 16 14:09:17.693735 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693531    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-2pv78qqtol62t\""
Sep 16 14:09:17.693735 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693598    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 16 14:09:17.693735 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693692    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
Sep 16 14:09:17.693862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.693816    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-fvkzs\""
Sep 16 14:09:17.703367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.703343    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6488dffc95-pksr7"]
Sep 16 14:09:17.761708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-client-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761710    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b39ebf5-2a88-49a0-a0f7-833344564b2f-audit-log\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761740    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-metrics-server-audit-profiles\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761765    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-tls\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761783    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761813    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-client-certs\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.761934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.761835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r9wnc\" (UniqueName: \"kubernetes.io/projected/5b39ebf5-2a88-49a0-a0f7-833344564b2f-kube-api-access-r9wnc\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-client-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862132    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b39ebf5-2a88-49a0-a0f7-833344564b2f-audit-log\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-metrics-server-audit-profiles\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862200    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-tls\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862224    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-client-certs\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862519 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862411    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r9wnc\" (UniqueName: \"kubernetes.io/projected/5b39ebf5-2a88-49a0-a0f7-833344564b2f-kube-api-access-r9wnc\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.862672 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.862649    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/5b39ebf5-2a88-49a0-a0f7-833344564b2f-audit-log\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.863056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.863033    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.863402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.863380    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/5b39ebf5-2a88-49a0-a0f7-833344564b2f-metrics-server-audit-profiles\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.864719 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.864693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-client-certs\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.864816 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.864789    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-client-ca-bundle\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.864816 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.864796    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/5b39ebf5-2a88-49a0-a0f7-833344564b2f-secret-metrics-server-tls\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.870927 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.870907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r9wnc\" (UniqueName: \"kubernetes.io/projected/5b39ebf5-2a88-49a0-a0f7-833344564b2f-kube-api-access-r9wnc\") pod \"metrics-server-6488dffc95-pksr7\" (UID: \"5b39ebf5-2a88-49a0-a0f7-833344564b2f\") " pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:17.999036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:17.999001    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:18.269211 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:18.269127    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6488dffc95-pksr7"]
Sep 16 14:09:18.273813 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:18.273786    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b39ebf5_2a88_49a0_a0f7_833344564b2f.slice/crio-87eb94c3e4c92163e9dc33a8d1d540f19a4474b02c1da05e87e83a8e0a078fa4 WatchSource:0}: Error finding container 87eb94c3e4c92163e9dc33a8d1d540f19a4474b02c1da05e87e83a8e0a078fa4: Status 404 returned error can't find the container with id 87eb94c3e4c92163e9dc33a8d1d540f19a4474b02c1da05e87e83a8e0a078fa4
Sep 16 14:09:18.517340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:18.517306    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7" event={"ID":"5b39ebf5-2a88-49a0-a0f7-833344564b2f","Type":"ContainerStarted","Data":"87eb94c3e4c92163e9dc33a8d1d540f19a4474b02c1da05e87e83a8e0a078fa4"}
Sep 16 14:09:19.236192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.236129    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:19.236192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.236193    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:19.241610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.241583    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:19.491682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.491575    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:09:19.494834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.494804    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.498014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.497712    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 14:09:19.498014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.497766    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 14:09:19.498014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.497790    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 14:09:19.498014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.497723    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 14:09:19.498014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498017    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 14:09:19.498370 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498082    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 14:09:19.498370 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498216    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4cgtr\""
Sep 16 14:09:19.499117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498871    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 14:09:19.499117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 14:09:19.499117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498956    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 14:09:19.499117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498959    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 14:09:19.499117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.498880    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 14:09:19.499412 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.499226    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 14:09:19.499471 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.499451    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 14:09:19.500470 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.500450    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 14:09:19.500675 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.500654    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e01mmn37emtj8\""
Sep 16 14:09:19.517309 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.517287    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:09:19.525350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.525324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:19.577767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.577767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577806    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577827    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.577992    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578038    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578071    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578119    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578177    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578234    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578260    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578336    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578368    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n2js7\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578431    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.578664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.578457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679499    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679548    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679598    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679621    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679881 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679881 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679815    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.679881 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679855    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679903    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.679981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680014    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680041    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680094    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680136    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680162    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680192    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n2js7\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680219    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680681    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.680908 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.680886    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.681251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.681228    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.681610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.681585    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.681708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.681607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.682173 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.681874    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.683032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.682945    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.683313 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.683127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.683313 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.683211    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.684445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.684386    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.685970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686056    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686208 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686127    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686431 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686410    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686653    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.686773 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.686658    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.687116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.687096    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.687863 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.687847    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.693985 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.693967    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n2js7\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7\") pod \"prometheus-k8s-0\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:19.808845 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:19.808759    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:20.327591 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.327488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:09:20.331053 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:09:20.331022    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod654d26ea_2971_495c_9028_042554b95d9d.slice/crio-cb1aff953ea58c33fc850390b5f170ca69fe6f2f5f62758476ea9721c966fdf5 WatchSource:0}: Error finding container cb1aff953ea58c33fc850390b5f170ca69fe6f2f5f62758476ea9721c966fdf5: Status 404 returned error can't find the container with id cb1aff953ea58c33fc850390b5f170ca69fe6f2f5f62758476ea9721c966fdf5
Sep 16 14:09:20.526388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.526350    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7" exitCode=0
Sep 16 14:09:20.526539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.526450    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"}
Sep 16 14:09:20.526539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.526523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"cb1aff953ea58c33fc850390b5f170ca69fe6f2f5f62758476ea9721c966fdf5"}
Sep 16 14:09:20.528135 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.528081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7" event={"ID":"5b39ebf5-2a88-49a0-a0f7-833344564b2f","Type":"ContainerStarted","Data":"8b3574be4115c6f2a0cb2667d87b0b7356d87c2b8d31953c8991f5775ceb403c"}
Sep 16 14:09:20.529754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.529702    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7" exitCode=0
Sep 16 14:09:20.529754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.529740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"}
Sep 16 14:09:20.575277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:20.575228    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7" podStartSLOduration=1.799069921 podStartE2EDuration="3.575213815s" totalImagesPullingTime="1.776143894s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:20.573725784 +0000 UTC m=+81.980306006" watchObservedRunningTime="2026-09-16 14:09:20.575213815 +0000 UTC m=+81.981794014"
Sep 16 14:09:24.554196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.554153    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"}
Sep 16 14:09:24.554622 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.554286    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"}
Sep 16 14:09:24.554622 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.554335    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"}
Sep 16 14:09:24.557028 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.557001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"}
Sep 16 14:09:24.557135 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.557060    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"}
Sep 16 14:09:24.572652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.572597    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" podUID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" containerName="registry" containerID="cri-o://e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f" gracePeriod=30
Sep 16 14:09:24.816373 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.816348    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:09:24.928493 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928459    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928509    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928531    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928591    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928648    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928943 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928682    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.928943 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cvdxt\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt\") pod \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\" (UID: \"7e68ab3c-d6b6-4498-a266-f390ed6ca2de\") "
Sep 16 14:09:24.929051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.928944    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:24.929115 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.929055    2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-certificates\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:24.929527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.929497    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:24.931504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.931453    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:09:24.931504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.931473    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt" (OuterVolumeSpecName: "kube-api-access-cvdxt") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "kube-api-access-cvdxt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:09:24.931661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.931574    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:09:24.931841 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.931816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:09:24.932145 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.932078    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:09:24.940399 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:24.940368    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "7e68ab3c-d6b6-4498-a266-f390ed6ca2de" (UID: "7e68ab3c-d6b6-4498-a266-f390ed6ca2de"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:09:25.030734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030652    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cvdxt\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-kube-api-access-cvdxt\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.030734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030692    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-trusted-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.030734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030710    2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-image-registry-private-configuration\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.030734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030735    2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-installation-pull-secrets\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.031114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030749    2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-bound-sa-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.031114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030763    2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-registry-tls\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.031114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.030776    2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/7e68ab3c-d6b6-4498-a266-f390ed6ca2de-ca-trust-extracted\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:25.567258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.567065    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"}
Sep 16 14:09:25.567258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.567157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"}
Sep 16 14:09:25.569149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.568611    2580 generic.go:356] "Generic (PLEG): container finished" podID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" containerID="e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f" exitCode=0
Sep 16 14:09:25.569149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.568790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk"
Sep 16 14:09:25.569384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.569352    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" event={"ID":"7e68ab3c-d6b6-4498-a266-f390ed6ca2de","Type":"ContainerDied","Data":"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"}
Sep 16 14:09:25.569483 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.569428    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-758dbd5bcc-4rmfk" event={"ID":"7e68ab3c-d6b6-4498-a266-f390ed6ca2de","Type":"ContainerDied","Data":"0c81947853b43d15304d7211d6ec3cc37873bc6107eb82dbdff747298c977a4f"}
Sep 16 14:09:25.569483 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.569474    2580 scope.go:117] "RemoveContainer" containerID="e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"
Sep 16 14:09:25.591196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.591164    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:09:25.592439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.592406    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-758dbd5bcc-4rmfk"]
Sep 16 14:09:25.646955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.646926    2580 scope.go:117] "RemoveContainer" containerID="e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"
Sep 16 14:09:25.647320 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:09:25.647299    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f\": container with ID starting with e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f not found: ID does not exist" containerID="e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"
Sep 16 14:09:25.647450 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:25.647330    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f"} err="failed to get container status \"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f\": rpc error: code = NotFound desc = could not find container \"e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f\": container with ID starting with e13727e7466c36283d87f9e699473413bbb96d0eef29e724692a2fcda25ea83f not found: ID does not exist"
Sep 16 14:09:26.576543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.576503    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"}
Sep 16 14:09:26.577039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.576632    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"}
Sep 16 14:09:26.577039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.576682    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"}
Sep 16 14:09:26.577039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.576725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerStarted","Data":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"}
Sep 16 14:09:26.581298 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.581268    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerStarted","Data":"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"}
Sep 16 14:09:26.617353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.617270    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=1.993284047 podStartE2EDuration="7.617248693s" totalImagesPullingTime="5.604885353s" totalInitContainerRuntime="19.079293ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:26.615425535 +0000 UTC m=+88.022005742" watchObservedRunningTime="2026-09-16 14:09:26.617248693 +0000 UTC m=+88.023828892"
Sep 16 14:09:26.646234 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:26.646182    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.461329722 podStartE2EDuration="12.646162145s" totalImagesPullingTime="9.168549033s" totalInitContainerRuntime="16.28339ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:09:14 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:09:26.643849609 +0000 UTC m=+88.050429850" watchObservedRunningTime="2026-09-16 14:09:26.646162145 +0000 UTC m=+88.052742353"
Sep 16 14:09:27.182752 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:27.182712    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" path="/var/lib/kubelet/pods/7e68ab3c-d6b6-4498-a266-f390ed6ca2de/volumes"
Sep 16 14:09:29.809835 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:29.809795    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:09:34.374081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:34.374047    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:09:37.999685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:37.999632    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:38.000087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:37.999715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:41.485616 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:41.485579    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-xx49p"
Sep 16 14:09:58.005375 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:58.005347    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:58.009500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:58.009471    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6488dffc95-pksr7"
Sep 16 14:09:59.393080 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.392958    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-76f5dd85db-crxzz" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerName="console" containerID="cri-o://40473dc0d87e35c108ddbb19faf54a4e2854b64e52b175845679e72ed8792db6" gracePeriod=15
Sep 16 14:09:59.522598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.522527    2580 patch_prober.go:28] interesting pod/console-76f5dd85db-crxzz container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.133.0.12:8443/health\": dial tcp 10.133.0.12:8443: connect: connection refused" start-of-body=
Sep 16 14:09:59.523248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.522860    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-76f5dd85db-crxzz" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerName="console" probeResult="failure" output="Get \"https://10.133.0.12:8443/health\": dial tcp 10.133.0.12:8443: connect: connection refused"
Sep 16 14:09:59.703749 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.703719    2580 generic.go:356] "Generic (PLEG): container finished" podID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerID="40473dc0d87e35c108ddbb19faf54a4e2854b64e52b175845679e72ed8792db6" exitCode=0
Sep 16 14:09:59.703901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.703797    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5dd85db-crxzz" event={"ID":"4fba751b-173d-4b6b-8d09-c4d57e7fa045","Type":"ContainerDied","Data":"40473dc0d87e35c108ddbb19faf54a4e2854b64e52b175845679e72ed8792db6"}
Sep 16 14:09:59.703901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.703866    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-76f5dd85db-crxzz" event={"ID":"4fba751b-173d-4b6b-8d09-c4d57e7fa045","Type":"ContainerDied","Data":"3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb"}
Sep 16 14:09:59.703901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.703889    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3e77b242d8d5b3ced6d4f8d6721bc0f4984f57e02d0f8fadb4d05c3a0bc78dbb"
Sep 16 14:09:59.705480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.705464    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:09:59.708600 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708574    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708606    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708628    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708691    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708726    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708760    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6mwct\" (UniqueName: \"kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.708942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.708786    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle\") pod \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\" (UID: \"4fba751b-173d-4b6b-8d09-c4d57e7fa045\") "
Sep 16 14:09:59.709410 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.709355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config" (OuterVolumeSpecName: "console-config") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:59.709410 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.709389    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca" (OuterVolumeSpecName: "service-ca") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:59.709410 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.709400    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:59.709657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.709538    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:09:59.711316 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.711270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct" (OuterVolumeSpecName: "kube-api-access-6mwct") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "kube-api-access-6mwct". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:09:59.711316 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.711291    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:09:59.711441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.711356    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4fba751b-173d-4b6b-8d09-c4d57e7fa045" (UID: "4fba751b-173d-4b6b-8d09-c4d57e7fa045"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:09:59.809364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809331    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6mwct\" (UniqueName: \"kubernetes.io/projected/4fba751b-173d-4b6b-8d09-c4d57e7fa045-kube-api-access-6mwct\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809359    2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-trusted-ca-bundle\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809368    2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-oauth-serving-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809378    2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809387    2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-oauth-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809396    2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-serving-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:09:59.809695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:09:59.809407    2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4fba751b-173d-4b6b-8d09-c4d57e7fa045-console-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:00.708279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:00.708245    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-76f5dd85db-crxzz"
Sep 16 14:10:00.730632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:00.730597    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:10:00.735515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:00.735482    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-76f5dd85db-crxzz"]
Sep 16 14:10:01.182889 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:01.182811    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" path="/var/lib/kubelet/pods/4fba751b-173d-4b6b-8d09-c4d57e7fa045/volumes"
Sep 16 14:10:19.809386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:19.809338    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:19.828816 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:19.828789    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:20.796010 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:20.795980    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:33.507921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.507883    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:33.508455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508217    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="alertmanager" containerID="cri-o://65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941" gracePeriod=120
Sep 16 14:10:33.508455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508282    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-web" containerID="cri-o://10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd" gracePeriod=120
Sep 16 14:10:33.508455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508309    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="config-reloader" containerID="cri-o://ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4" gracePeriod=120
Sep 16 14:10:33.508455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508275    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-metric" containerID="cri-o://d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536" gracePeriod=120
Sep 16 14:10:33.508455 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508352    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy" containerID="cri-o://8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d" gracePeriod=120
Sep 16 14:10:33.508763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.508330    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="prom-label-proxy" containerID="cri-o://bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac" gracePeriod=120
Sep 16 14:10:33.830155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830057    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac" exitCode=0
Sep 16 14:10:33.830155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830081    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d" exitCode=0
Sep 16 14:10:33.830155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830089    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4" exitCode=0
Sep 16 14:10:33.830155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830096    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941" exitCode=0
Sep 16 14:10:33.830155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"}
Sep 16 14:10:33.830404 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"}
Sep 16 14:10:33.830404 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"}
Sep 16 14:10:33.830404 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:33.830247    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"}
Sep 16 14:10:34.742395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.742372    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:34.837782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837691    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536" exitCode=0
Sep 16 14:10:34.837782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837722    2580 generic.go:356] "Generic (PLEG): container finished" podID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerID="10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd" exitCode=0
Sep 16 14:10:34.837782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"}
Sep 16 14:10:34.837782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837756    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:34.838045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837822    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"}
Sep 16 14:10:34.838045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837864    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"be394e4f-bf76-43c3-8d4a-d2ec54563483","Type":"ContainerDied","Data":"cd9be160d1594970239d35b4636cccbe323ea410d560f4c9191b9278c920d75a"}
Sep 16 14:10:34.838045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.837896    2580 scope.go:117] "RemoveContainer" containerID="bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"
Sep 16 14:10:34.845725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.845701    2580 scope.go:117] "RemoveContainer" containerID="d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"
Sep 16 14:10:34.853042 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.853019    2580 scope.go:117] "RemoveContainer" containerID="8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"
Sep 16 14:10:34.859474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.859454    2580 scope.go:117] "RemoveContainer" containerID="10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"
Sep 16 14:10:34.865912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.865895    2580 scope.go:117] "RemoveContainer" containerID="ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"
Sep 16 14:10:34.872251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.872233    2580 scope.go:117] "RemoveContainer" containerID="65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"
Sep 16 14:10:34.878657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.878643    2580 scope.go:117] "RemoveContainer" containerID="6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"
Sep 16 14:10:34.885061 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885044    2580 scope.go:117] "RemoveContainer" containerID="bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"
Sep 16 14:10:34.885327 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.885308    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac\": container with ID starting with bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac not found: ID does not exist" containerID="bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"
Sep 16 14:10:34.885395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885340    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"} err="failed to get container status \"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac\": rpc error: code = NotFound desc = could not find container \"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac\": container with ID starting with bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac not found: ID does not exist"
Sep 16 14:10:34.885395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885368    2580 scope.go:117] "RemoveContainer" containerID="d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"
Sep 16 14:10:34.885629 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.885607    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536\": container with ID starting with d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536 not found: ID does not exist" containerID="d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"
Sep 16 14:10:34.885711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885633    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"} err="failed to get container status \"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536\": rpc error: code = NotFound desc = could not find container \"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536\": container with ID starting with d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536 not found: ID does not exist"
Sep 16 14:10:34.885711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885649    2580 scope.go:117] "RemoveContainer" containerID="8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"
Sep 16 14:10:34.885895 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.885878    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d\": container with ID starting with 8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d not found: ID does not exist" containerID="8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"
Sep 16 14:10:34.885956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885924    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"} err="failed to get container status \"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d\": rpc error: code = NotFound desc = could not find container \"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d\": container with ID starting with 8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d not found: ID does not exist"
Sep 16 14:10:34.885956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.885948    2580 scope.go:117] "RemoveContainer" containerID="10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"
Sep 16 14:10:34.886187 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.886171    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd\": container with ID starting with 10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd not found: ID does not exist" containerID="10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"
Sep 16 14:10:34.886227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886192    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"} err="failed to get container status \"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd\": rpc error: code = NotFound desc = could not find container \"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd\": container with ID starting with 10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd not found: ID does not exist"
Sep 16 14:10:34.886227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886204    2580 scope.go:117] "RemoveContainer" containerID="ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"
Sep 16 14:10:34.886419 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.886404    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4\": container with ID starting with ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4 not found: ID does not exist" containerID="ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"
Sep 16 14:10:34.886481 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886428    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"} err="failed to get container status \"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4\": rpc error: code = NotFound desc = could not find container \"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4\": container with ID starting with ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4 not found: ID does not exist"
Sep 16 14:10:34.886481 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886450    2580 scope.go:117] "RemoveContainer" containerID="65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"
Sep 16 14:10:34.886702 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.886685    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941\": container with ID starting with 65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941 not found: ID does not exist" containerID="65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"
Sep 16 14:10:34.886744 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886706    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"} err="failed to get container status \"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941\": rpc error: code = NotFound desc = could not find container \"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941\": container with ID starting with 65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941 not found: ID does not exist"
Sep 16 14:10:34.886744 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886722    2580 scope.go:117] "RemoveContainer" containerID="6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"
Sep 16 14:10:34.886947 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:34.886932    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7\": container with ID starting with 6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7 not found: ID does not exist" containerID="6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"
Sep 16 14:10:34.886995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886950    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"} err="failed to get container status \"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7\": rpc error: code = NotFound desc = could not find container \"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7\": container with ID starting with 6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7 not found: ID does not exist"
Sep 16 14:10:34.886995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.886962    2580 scope.go:117] "RemoveContainer" containerID="bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"
Sep 16 14:10:34.887153 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887138    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac"} err="failed to get container status \"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac\": rpc error: code = NotFound desc = could not find container \"bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac\": container with ID starting with bf122851fbfa1bfc257b51bf7f09f387de19b8c0570f8828bd3bd7c5df1662ac not found: ID does not exist"
Sep 16 14:10:34.887195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887152    2580 scope.go:117] "RemoveContainer" containerID="d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"
Sep 16 14:10:34.887382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887366    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536"} err="failed to get container status \"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536\": rpc error: code = NotFound desc = could not find container \"d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536\": container with ID starting with d6c10e219246bdefadb5cedc59d5acc33a34f2f9ee782f824569eb8da08e5536 not found: ID does not exist"
Sep 16 14:10:34.887418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887383    2580 scope.go:117] "RemoveContainer" containerID="8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"
Sep 16 14:10:34.887648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887597    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d"} err="failed to get container status \"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d\": rpc error: code = NotFound desc = could not find container \"8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d\": container with ID starting with 8d5389e412137075ec134b2bdba05fc67249ba67b81e44f375a039f8f33e966d not found: ID does not exist"
Sep 16 14:10:34.887648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887613    2580 scope.go:117] "RemoveContainer" containerID="10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"
Sep 16 14:10:34.887847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887829    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd"} err="failed to get container status \"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd\": rpc error: code = NotFound desc = could not find container \"10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd\": container with ID starting with 10b85e7c0cbc9609dcfc9e6be67e77304ea23bd6d055fba9cf65083bad0158dd not found: ID does not exist"
Sep 16 14:10:34.887883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.887849    2580 scope.go:117] "RemoveContainer" containerID="ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"
Sep 16 14:10:34.888078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.888061    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4"} err="failed to get container status \"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4\": rpc error: code = NotFound desc = could not find container \"ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4\": container with ID starting with ce68ecbdcdcc8a294deb1ce97e6298aa1945280a1286cf58f33920846de394e4 not found: ID does not exist"
Sep 16 14:10:34.888117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.888079    2580 scope.go:117] "RemoveContainer" containerID="65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"
Sep 16 14:10:34.888301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.888280    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941"} err="failed to get container status \"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941\": rpc error: code = NotFound desc = could not find container \"65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941\": container with ID starting with 65e3f1b726d00e183d3eac93e85a8aba1476642ad30c9d5c02efbd8fc1d9c941 not found: ID does not exist"
Sep 16 14:10:34.888301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.888301    2580 scope.go:117] "RemoveContainer" containerID="6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"
Sep 16 14:10:34.888546 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.888526    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7"} err="failed to get container status \"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7\": rpc error: code = NotFound desc = could not find container \"6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7\": container with ID starting with 6e5dbaf0ac9a22f670280ab9e09ce7f6fb2c2a7751254719c86e1ceec4764ac7 not found: ID does not exist"
Sep 16 14:10:34.893795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893779    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.893851 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvkdh\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.893851 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893834    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.893915 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893852    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.893915 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893874    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893918    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893939    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.893970    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894003    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894020    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894045    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894091    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets\") pod \"be394e4f-bf76-43c3-8d4a-d2ec54563483\" (UID: \"be394e4f-bf76-43c3-8d4a-d2ec54563483\") "
Sep 16 14:10:34.894381 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894167    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:34.894381 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894338    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-main-db\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.895015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894521    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:34.895015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.894519    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:34.896492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.896464    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.896492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.896467    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh" (OuterVolumeSpecName: "kube-api-access-qvkdh") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "kube-api-access-qvkdh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:34.896786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.896760    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.896898 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.896801    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.897211 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.897176    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume" (OuterVolumeSpecName: "config-volume") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.897965 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.897946    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out" (OuterVolumeSpecName: "config-out") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:34.898050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.897963    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:34.898050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.897949    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.902515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.902491    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.907966 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.907948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config" (OuterVolumeSpecName: "web-config") pod "be394e4f-bf76-43c3-8d4a-d2ec54563483" (UID: "be394e4f-bf76-43c3-8d4a-d2ec54563483"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:34.995341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995306    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-web-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995337    2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995348    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995359    2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995367    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/be394e4f-bf76-43c3-8d4a-d2ec54563483-config-out\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995375    2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-cluster-tls-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995383    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-tls-assets\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995392    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995400    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvkdh\" (UniqueName: \"kubernetes.io/projected/be394e4f-bf76-43c3-8d4a-d2ec54563483-kube-api-access-qvkdh\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995409    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-main-tls\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995417    2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/be394e4f-bf76-43c3-8d4a-d2ec54563483-metrics-client-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:34.995576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:34.995426    2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/be394e4f-bf76-43c3-8d4a-d2ec54563483-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:35.168655 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.168621    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:35.174713 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.174691    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:35.182694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.182671    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" path="/var/lib/kubelet/pods/be394e4f-bf76-43c3-8d4a-d2ec54563483/volumes"
Sep 16 14:10:35.223365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223337    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:35.223641 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223627    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="init-config-reloader"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223645    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="init-config-reloader"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223656    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="config-reloader"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223663    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="config-reloader"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223671    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerName="console"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223676    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerName="console"
Sep 16 14:10:35.223682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223684    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-web"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223691    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-web"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223704    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="prom-label-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223709    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="prom-label-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223715    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="alertmanager"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223720    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="alertmanager"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223725    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-metric"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223730    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-metric"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223739    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" containerName="registry"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223744    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" containerName="registry"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223749    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223754    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223808    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="alertmanager"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223816    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223821    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-metric"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223828    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="kube-rbac-proxy-web"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223833    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="prom-label-proxy"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223838    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e68ab3c-d6b6-4498-a266-f390ed6ca2de" containerName="registry"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223844    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4fba751b-173d-4b6b-8d09-c4d57e7fa045" containerName="console"
Sep 16 14:10:35.223857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.223855    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be394e4f-bf76-43c3-8d4a-d2ec54563483" containerName="config-reloader"
Sep 16 14:10:35.227257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.227242    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.234659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234635    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
Sep 16 14:10:35.234766 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234637    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
Sep 16 14:10:35.234766 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234639    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
Sep 16 14:10:35.234766 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234637    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
Sep 16 14:10:35.234920 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234679    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
Sep 16 14:10:35.234920 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234681    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
Sep 16 14:10:35.234920 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.234680    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
Sep 16 14:10:35.235278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.235263    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-gbwfg\""
Sep 16 14:10:35.235352 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.235327    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
Sep 16 14:10:35.255284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.255236    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
Sep 16 14:10:35.275538 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.275507    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:35.297397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297367    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297403    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297426    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297491    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-web-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297617    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297643    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297660    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsbwp\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-kube-api-access-tsbwp\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297690    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297719    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-out\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297735    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297752    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.297815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.297772    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398551    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398653    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-web-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398683    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398741    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398780    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsbwp\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-kube-api-access-tsbwp\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398832    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398860    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-out\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398886    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398912    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.398948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398945    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.399327 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.398983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.399327 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.399012    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.399487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.399461    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.399598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.399543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.399756 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.399678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5ba33eff-0102-47bd-9fa7-06fc76a6df17-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.401919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.401895    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-volume\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.401919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.401915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402085 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.401942    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-tls-assets\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402085 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.402021    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.402214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/5ba33eff-0102-47bd-9fa7-06fc76a6df17-config-out\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402291 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.402274    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.402340    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.402465 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.402446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-web-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.403647 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.403623    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/5ba33eff-0102-47bd-9fa7-06fc76a6df17-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.407654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.407637    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsbwp\" (UniqueName: \"kubernetes.io/projected/5ba33eff-0102-47bd-9fa7-06fc76a6df17-kube-api-access-tsbwp\") pod \"alertmanager-main-0\" (UID: \"5ba33eff-0102-47bd-9fa7-06fc76a6df17\") " pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.536499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.536472    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
Sep 16 14:10:35.816407 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.816370    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
Sep 16 14:10:35.819616 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:10:35.819585    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ba33eff_0102_47bd_9fa7_06fc76a6df17.slice/crio-1c31596e48eff0ba2113f2b19e25963ad1d98ce68c9de894463080871efeaae6 WatchSource:0}: Error finding container 1c31596e48eff0ba2113f2b19e25963ad1d98ce68c9de894463080871efeaae6: Status 404 returned error can't find the container with id 1c31596e48eff0ba2113f2b19e25963ad1d98ce68c9de894463080871efeaae6
Sep 16 14:10:35.843589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:35.843546    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"1c31596e48eff0ba2113f2b19e25963ad1d98ce68c9de894463080871efeaae6"}
Sep 16 14:10:36.854288 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:36.854191    2580 generic.go:356] "Generic (PLEG): container finished" podID="5ba33eff-0102-47bd-9fa7-06fc76a6df17" containerID="6b65b5f4a6bc2dfc5edfbcffa95909926bcd8738437d31fab3508bcb00f36f50" exitCode=0
Sep 16 14:10:36.854288 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:36.854238    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerDied","Data":"6b65b5f4a6bc2dfc5edfbcffa95909926bcd8738437d31fab3508bcb00f36f50"}
Sep 16 14:10:37.610190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.610133    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"]
Sep 16 14:10:37.612309 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.612289    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.616047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.616023    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
Sep 16 14:10:37.616456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.616439    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-fllpv\""
Sep 16 14:10:37.616718 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.616699    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
Sep 16 14:10:37.617299 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.617279    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
Sep 16 14:10:37.617648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.617627    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
Sep 16 14:10:37.617956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.617637    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
Sep 16 14:10:37.622999 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.622977    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
Sep 16 14:10:37.630661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.630641    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"]
Sep 16 14:10:37.716866 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.716826    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-federate-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.716866 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.716866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.716909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-metrics-client-ca\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.716936    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-serving-certs-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.716956    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rblgp\" (UniqueName: \"kubernetes.io/projected/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-kube-api-access-rblgp\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.717017    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.717067    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.717116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.717099    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818291 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-federate-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818291 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818293    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818415    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-metrics-client-ca\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818451    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-serving-certs-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rblgp\" (UniqueName: \"kubernetes.io/projected/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-kube-api-access-rblgp\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818498    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818546    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.818739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.818611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.819232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.819157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-serving-certs-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.819361 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.819237    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.819361 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.819308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-metrics-client-ca\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.821097 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.821067    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.821189 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.821107    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-secret-telemeter-client\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.821341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.821323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-federate-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.821394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.821375    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-telemeter-client-tls\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.830643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.830618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rblgp\" (UniqueName: \"kubernetes.io/projected/cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909-kube-api-access-rblgp\") pod \"telemeter-client-7f6758f99d-lvm6q\" (UID: \"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909\") " pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:37.861492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861415    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"ae30c2e43098df574612d7f602a3ab4ee22b0ef3043ad189ce42775c9f47c451"}
Sep 16 14:10:37.861825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861499    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"fb60175a1b300238b4a40d1e0382a18a0db9a1089e4319f83a94aa4ac1f7c637"}
Sep 16 14:10:37.861825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"1069a95c782ce3f56436caba1203f0daaf372af4c26f5146fed50aca14fc326d"}
Sep 16 14:10:37.861825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861599    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"676b9ca7a2419aae07a2048f319bd4596b0e9b18759b147de6dae869d9f19e6a"}
Sep 16 14:10:37.861825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"74bbf3f303276f1e9d444af5f1dc227e0b2512b37e7b39bda602852eb283a7ec"}
Sep 16 14:10:37.861825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.861674    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"5ba33eff-0102-47bd-9fa7-06fc76a6df17","Type":"ContainerStarted","Data":"f9e337790e1f84790bedba0756500ece25acf130cfa7513e0aa1501be81208bd"}
Sep 16 14:10:37.909749 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.909696    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.8942773539999997 podStartE2EDuration="2.909678547s" totalImagesPullingTime="0s" totalInitContainerRuntime="15.401193ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:37.909409297 +0000 UTC m=+159.315989527" watchObservedRunningTime="2026-09-16 14:10:37.909678547 +0000 UTC m=+159.316258755"
Sep 16 14:10:37.918666 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918635    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:37.918973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918935    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="prometheus" containerID="cri-o://3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4" gracePeriod=600
Sep 16 14:10:37.918973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918947    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy" containerID="cri-o://b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d" gracePeriod=600
Sep 16 14:10:37.919152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.919006    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-thanos" containerID="cri-o://bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72" gracePeriod=600
Sep 16 14:10:37.919152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918952    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="thanos-sidecar" containerID="cri-o://8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1" gracePeriod=600
Sep 16 14:10:37.919152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918987    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-web" containerID="cri-o://8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e" gracePeriod=600
Sep 16 14:10:37.919306 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.918988    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="config-reloader" containerID="cri-o://39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea" gracePeriod=600
Sep 16 14:10:37.922105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:37.922007    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"
Sep 16 14:10:38.150162 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.150138    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221245    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221341    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221365    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221397    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221427    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221475    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221499    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221522    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221550    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221607    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221633    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221659    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n2js7\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221684    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221742    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221775    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.222499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221802    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.223477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221832    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.223477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.221872    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls\") pod \"654d26ea-2971-495c-9028-042554b95d9d\" (UID: \"654d26ea-2971-495c-9028-042554b95d9d\") "
Sep 16 14:10:38.223477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.223173    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.223836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.223593    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.224575 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.224538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7f6758f99d-lvm6q"]
Sep 16 14:10:38.224575 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.224545    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.224892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.224869    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.227783 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.227355    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.229406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.228927    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:38.229406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.229002    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.229568 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.229476    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.229800 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.229768    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:10:38.230220 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.229984    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.231342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231194    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out" (OuterVolumeSpecName: "config-out") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:10:38.231342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231199    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7" (OuterVolumeSpecName: "kube-api-access-n2js7") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "kube-api-access-n2js7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:38.231342 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:10:38.231208    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcbdbf2f6_2ce5_45e2_a152_6d6c5ea8e909.slice/crio-99b376be58242ed8c50deefcb8995a57f4f56f20a07d1d3e2eaecdc3926337ea WatchSource:0}: Error finding container 99b376be58242ed8c50deefcb8995a57f4f56f20a07d1d3e2eaecdc3926337ea: Status 404 returned error can't find the container with id 99b376be58242ed8c50deefcb8995a57f4f56f20a07d1d3e2eaecdc3926337ea
Sep 16 14:10:38.231342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231260    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config" (OuterVolumeSpecName: "config") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.231342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231303    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.231681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231416    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.232079 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.231990    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.232166 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.232089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:10:38.232468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.232424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.232796 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.232761    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.244168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.244115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config" (OuterVolumeSpecName: "web-config") pod "654d26ea-2971-495c-9028-042554b95d9d" (UID: "654d26ea-2971-495c-9028-042554b95d9d"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:10:38.322861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322826    2580 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-metrics-client-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.322861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322858    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322870    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322882    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322891    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322901    2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-web-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322910    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-metrics-client-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322919    2580 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-thanos-prometheus-http-client-file\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322929    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n2js7\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-kube-api-access-n2js7\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322937    2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-config-out\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322945    2580 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-prometheus-k8s-tls\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322953    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-trusted-ca-bundle\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322962    2580 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-config\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322970    2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/654d26ea-2971-495c-9028-042554b95d9d-tls-assets\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322979    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322986    2580 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-grpc-tls\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.322994    2580 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.323002    2580 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/654d26ea-2971-495c-9028-042554b95d9d-secret-kube-rbac-proxy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.323010    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-db\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.323058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.323018    2580 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/654d26ea-2971-495c-9028-042554b95d9d-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870498    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72" exitCode=0
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870532    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d" exitCode=0
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870542    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e" exitCode=0
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870552    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1" exitCode=0
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870577    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea" exitCode=0
Sep 16 14:10:38.870599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870584    2580 generic.go:356] "Generic (PLEG): container finished" podID="654d26ea-2971-495c-9028-042554b95d9d" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4" exitCode=0
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870629    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870769    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870810    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870795    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870852    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870893    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"}
Sep 16 14:10:38.871209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.870933    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"654d26ea-2971-495c-9028-042554b95d9d","Type":"ContainerDied","Data":"cb1aff953ea58c33fc850390b5f170ca69fe6f2f5f62758476ea9721c966fdf5"}
Sep 16 14:10:38.872697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.872520    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q" event={"ID":"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909","Type":"ContainerStarted","Data":"99b376be58242ed8c50deefcb8995a57f4f56f20a07d1d3e2eaecdc3926337ea"}
Sep 16 14:10:38.880953 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.880936    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.889023 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.889002    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.896821 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.896791    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.904506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.904471    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.904693 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.904670    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:38.908735 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.908713    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:38.912907 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.912888    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.921668 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.921496    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.929124 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.929107    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.929440 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.929415    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.929507 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.929453    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.929507 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.929482    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.929800 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.929778    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.929914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.929805    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.929914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.929822    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.930091 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.930069    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.930132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930101    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.930132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930121    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.930412 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.930385    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.930480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930418    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.930480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930438    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.930733 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.930717    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.930769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930737    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.930769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.930751    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.931001 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.930984    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.931067 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931009    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.931067 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931028    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.931267 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:10:38.931249    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.931306 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931274    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.931306 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931291    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.931607 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931548    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.931607 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931603    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.932028 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.931992    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.932028 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932018    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.932318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932290    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.932318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932308    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.932544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932524    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.932544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932545    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.932845 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932827    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.932845 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.932845    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.933092 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933077    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.933152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933093    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.933435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933415    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.933507 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933437    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.933716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933693    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.933770 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933718    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.933989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933971    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.933989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.933989    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.934195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934179    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.934246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934195    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.934400 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934381    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.934458 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934401    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.934624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934604    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.934708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934625    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.934849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934831    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.934914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.934852    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.935077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935061    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.935117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935077    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.935307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935290    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.935307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935306    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.935505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935490    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.935540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935505    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.935725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935710    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.935774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935727    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.936004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.935989    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.936065 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936005    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.936279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936262    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.936345 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936281    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.936539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936514    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.936637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936541    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.936868 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936847    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.936868 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.936867    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.937111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937092    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.937190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937112    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.937415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937380    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.937482 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937418    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.937701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937680    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.937771 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937701    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.937923 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937901    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.937979 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.937922    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.938191 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938170    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.938248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938192    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.938456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938435    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.938530 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938457    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.938726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938703    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.938856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.938726    2580 scope.go:117] "RemoveContainer" containerID="bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"
Sep 16 14:10:38.939091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939061    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72"} err="failed to get container status \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": rpc error: code = NotFound desc = could not find container \"bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72\": container with ID starting with bebad2e5c9fabd58902aee3379016bec7385461ec20d11b788a9b2a5722d0d72 not found: ID does not exist"
Sep 16 14:10:38.939162 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939091    2580 scope.go:117] "RemoveContainer" containerID="b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"
Sep 16 14:10:38.939411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939372    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d"} err="failed to get container status \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": rpc error: code = NotFound desc = could not find container \"b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d\": container with ID starting with b54a1a7269543b2d521ab4cc9bdfc58a7e72c7180021ce6d3fe25e39e5f5ec4d not found: ID does not exist"
Sep 16 14:10:38.939411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939393    2580 scope.go:117] "RemoveContainer" containerID="8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"
Sep 16 14:10:38.939706 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939678    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e"} err="failed to get container status \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": rpc error: code = NotFound desc = could not find container \"8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e\": container with ID starting with 8b5ad5c636be8792b8c42618d89754e0a9183468cf3e188f1b190e844e87908e not found: ID does not exist"
Sep 16 14:10:38.939783 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.939706    2580 scope.go:117] "RemoveContainer" containerID="8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"
Sep 16 14:10:38.940032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1"} err="failed to get container status \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": rpc error: code = NotFound desc = could not find container \"8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1\": container with ID starting with 8fdbbc63f8027f3aab835ac04a25bb24679d64c854d9a3b991e460e53473dcc1 not found: ID does not exist"
Sep 16 14:10:38.940083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940034    2580 scope.go:117] "RemoveContainer" containerID="39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"
Sep 16 14:10:38.940278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940256    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea"} err="failed to get container status \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": rpc error: code = NotFound desc = could not find container \"39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea\": container with ID starting with 39dce629638b39f25aad2d81a550bf55d098a25ebc3acc29ac710cd1d736c8ea not found: ID does not exist"
Sep 16 14:10:38.940327 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940279    2580 scope.go:117] "RemoveContainer" containerID="3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"
Sep 16 14:10:38.940546 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940520    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4"} err="failed to get container status \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": rpc error: code = NotFound desc = could not find container \"3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4\": container with ID starting with 3ec9a13f178d33a732c312ba38def4b214e46884ba275d01b1024e6274557dd4 not found: ID does not exist"
Sep 16 14:10:38.940716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940571    2580 scope.go:117] "RemoveContainer" containerID="0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"
Sep 16 14:10:38.940864 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.940839    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7"} err="failed to get container status \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": rpc error: code = NotFound desc = could not find container \"0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7\": container with ID starting with 0433c6be06b3f276173caa102f5de11851e7c283b04673d387f6044c8f3804a7 not found: ID does not exist"
Sep 16 14:10:38.950940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.950921    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:38.951270 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951255    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="init-config-reloader"
Sep 16 14:10:38.951321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951271    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="init-config-reloader"
Sep 16 14:10:38.951321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951286    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-web"
Sep 16 14:10:38.951321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951293    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-web"
Sep 16 14:10:38.951321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951306    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy"
Sep 16 14:10:38.951321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951315    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951325    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="config-reloader"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951330    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="config-reloader"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951337    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="prometheus"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951341    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="prometheus"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951347    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="thanos-sidecar"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951354    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="thanos-sidecar"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951364    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-thanos"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951371    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-thanos"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951429    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="thanos-sidecar"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951438    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-web"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951443    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy-thanos"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951450    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="config-reloader"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951456    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="prometheus"
Sep 16 14:10:38.951494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.951463    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="654d26ea-2971-495c-9028-042554b95d9d" containerName="kube-rbac-proxy"
Sep 16 14:10:38.954709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.954692    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:38.958956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.958932    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 16 14:10:38.959051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.958973    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 16 14:10:38.959051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.959025    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-e01mmn37emtj8\""
Sep 16 14:10:38.959223 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.959197    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 16 14:10:38.959573 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.959537    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 16 14:10:38.959678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.959660    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 16 14:10:38.959813 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.959796    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 16 14:10:38.960234 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.960217    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 16 14:10:38.960320 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.960257    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 16 14:10:38.960980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.960950    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 16 14:10:38.961269 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.961233    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 16 14:10:38.961366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.961346    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-4cgtr\""
Sep 16 14:10:38.962100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.962080    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 16 14:10:38.964193 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.964150    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 16 14:10:38.964827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.964805    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 16 14:10:38.968871 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.968854    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 16 14:10:38.976148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:38.976126    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.027866    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.027910    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.027967    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.027991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028015    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028046    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028091    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028112    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028233    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028263    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028287    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028351    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028430    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028457    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.038149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028479    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.039082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.039082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.039082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.028547    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.039082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.030982    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6bllm\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-kube-api-access-6bllm\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132082    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132128    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132146    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132173    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132203    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132311    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132327    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132349    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6bllm\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-kube-api-access-6bllm\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132400    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132422    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132468    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132490    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.132512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132514    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132604    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132627    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132849    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.132999    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133485 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.133191    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.133485 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.133405    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.134861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.134492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.134861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.134768    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.136024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.135619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.136257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.136230    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.136360 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.136309    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.137292 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.137246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.137613 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.137377    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.137613 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.137575    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config-out\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.137855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.137809    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.138681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.138657    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-web-config\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.139166 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.139141    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.139781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.139760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.139993 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.139970    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.139993 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.139983    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.143740 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.143720    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6bllm\" (UniqueName: \"kubernetes.io/projected/b0fed74a-cb53-4aef-befb-7bc5d2cd013d-kube-api-access-6bllm\") pod \"prometheus-k8s-0\" (UID: \"b0fed74a-cb53-4aef-befb-7bc5d2cd013d\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.183169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.183138    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="654d26ea-2971-495c-9028-042554b95d9d" path="/var/lib/kubelet/pods/654d26ea-2971-495c-9028-042554b95d9d/volumes"
Sep 16 14:10:39.266857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.266815    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:10:39.559585 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:39.558349    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 16 14:10:39.882215 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:10:39.882136    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb0fed74a_cb53_4aef_befb_7bc5d2cd013d.slice/crio-02f325007d20836e27d90999c335c4469f9bd59dce7e9349364f25878383394a WatchSource:0}: Error finding container 02f325007d20836e27d90999c335c4469f9bd59dce7e9349364f25878383394a: Status 404 returned error can't find the container with id 02f325007d20836e27d90999c335c4469f9bd59dce7e9349364f25878383394a
Sep 16 14:10:40.885500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.885457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q" event={"ID":"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909","Type":"ContainerStarted","Data":"8395a586b4b84e0924f7a7d00096b2c68bd1ba24db73283e8943b5b2906743f1"}
Sep 16 14:10:40.885957 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.885528    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q" event={"ID":"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909","Type":"ContainerStarted","Data":"b5c38abd3b704f63b0d4acc1eab880c39640ca763bf779f8ea0d0d7192af5632"}
Sep 16 14:10:40.885957 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.885573    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q" event={"ID":"cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909","Type":"ContainerStarted","Data":"bdad225e3354164dac86453bab07d2b7a5994edc45bc0601e1b5cfddaec22df7"}
Sep 16 14:10:40.887036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.887012    2580 generic.go:356] "Generic (PLEG): container finished" podID="b0fed74a-cb53-4aef-befb-7bc5d2cd013d" containerID="8b9e0b3314cb60f9c2c4a26aaf72c7740c33bd3ce7dd88ee23a33e504fbc4023" exitCode=0
Sep 16 14:10:40.887143 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.887076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerDied","Data":"8b9e0b3314cb60f9c2c4a26aaf72c7740c33bd3ce7dd88ee23a33e504fbc4023"}
Sep 16 14:10:40.887143 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.887127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"02f325007d20836e27d90999c335c4469f9bd59dce7e9349364f25878383394a"}
Sep 16 14:10:40.913756 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:40.913702    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7f6758f99d-lvm6q" podStartSLOduration=2.23936809 podStartE2EDuration="3.913684835s" totalImagesPullingTime="1.674316745s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:40.912437628 +0000 UTC m=+162.319017851" watchObservedRunningTime="2026-09-16 14:10:40.913684835 +0000 UTC m=+162.320265073"
Sep 16 14:10:41.894055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894016    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"e57569dc1b77a5035339842e90592c1802465f7811977cb23df7ee3bba3b9c3a"}
Sep 16 14:10:41.894441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"3f55b631a94cf9f21438e7ecacf399ff99e802cec2e0af8c1272368ee4eb24ca"}
Sep 16 14:10:41.894441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894140    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"09ecc218674cd2c75bd105fc20e8338e354686a7c52d1633166a7ff56f96219b"}
Sep 16 14:10:41.894441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894164    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"7779d84cc579e464aa37bba0c60aa8ac8b0c0aca18a90b6426a165d9e0f4c0ab"}
Sep 16 14:10:41.894441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894187    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"e7a7605703038c3fd0014a4e4e0b1ef4af41429cacb7bfbd375ce41cff32208f"}
Sep 16 14:10:41.894441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.894210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"b0fed74a-cb53-4aef-befb-7bc5d2cd013d","Type":"ContainerStarted","Data":"f9104679c5e0a431d2c13cc8961a4a274a0b8a5c382da5bf7c6f0ad11186f682"}
Sep 16 14:10:41.926627 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:41.926549    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.90464441 podStartE2EDuration="3.926525117s" totalImagesPullingTime="0s" totalInitContainerRuntime="21.880707ms" isStatefulPod=false podCreationTimestamp="2026-09-16 14:10:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:10:41.923893812 +0000 UTC m=+163.330474020" watchObservedRunningTime="2026-09-16 14:10:41.926525117 +0000 UTC m=+163.333105320"
Sep 16 14:10:44.267512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:10:44.267456    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:11:39.267270 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:11:39.267234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:11:39.282648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:11:39.282622    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:11:40.125125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:11:40.125100    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 16 14:12:58.977183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:12:58.977159    2580 kubelet.go:1648] "Image garbage collection succeeded"
Sep 16 14:13:30.508204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.508171    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"]
Sep 16 14:13:30.511423 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.511406    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.517081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.517045    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 16 14:13:30.517386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.517362    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-2dmtz\""
Sep 16 14:13:30.517552 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.517397    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 16 14:13:30.523653 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.523633    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"]
Sep 16 14:13:30.564485 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.564454    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6d72\" (UniqueName: \"kubernetes.io/projected/4b097f85-16ba-49f7-acda-a046bee40de0-kube-api-access-c6d72\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.564651 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.564502    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b097f85-16ba-49f7-acda-a046bee40de0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.665772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.665739    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c6d72\" (UniqueName: \"kubernetes.io/projected/4b097f85-16ba-49f7-acda-a046bee40de0-kube-api-access-c6d72\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.665934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.665789    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b097f85-16ba-49f7-acda-a046bee40de0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.666113 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.666097    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/4b097f85-16ba-49f7-acda-a046bee40de0-tmp\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.675603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.675551    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6d72\" (UniqueName: \"kubernetes.io/projected/4b097f85-16ba-49f7-acda-a046bee40de0-kube-api-access-c6d72\") pod \"openshift-lws-operator-fd8ccff4c-wmvfp\" (UID: \"4b097f85-16ba-49f7-acda-a046bee40de0\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:30.825859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:30.825757    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"
Sep 16 14:13:31.101621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:31.101526    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp"]
Sep 16 14:13:31.105686 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:13:31.105661    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4b097f85_16ba_49f7_acda_a046bee40de0.slice/crio-75809e5c14ed80495826555c7933e169fce5bb37fe010a71bf37aabc0694013c WatchSource:0}: Error finding container 75809e5c14ed80495826555c7933e169fce5bb37fe010a71bf37aabc0694013c: Status 404 returned error can't find the container with id 75809e5c14ed80495826555c7933e169fce5bb37fe010a71bf37aabc0694013c
Sep 16 14:13:31.107166 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:31.107150    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:13:31.523963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:31.523926    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp" event={"ID":"4b097f85-16ba-49f7-acda-a046bee40de0","Type":"ContainerStarted","Data":"75809e5c14ed80495826555c7933e169fce5bb37fe010a71bf37aabc0694013c"}
Sep 16 14:13:34.537441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:34.537407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp" event={"ID":"4b097f85-16ba-49f7-acda-a046bee40de0","Type":"ContainerStarted","Data":"ac1a801bf5aa8003ed5d57b3b79695b94049a4abdb816f8fe4f9119ab0d06bb1"}
Sep 16 14:13:34.555541 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:34.555489    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-wmvfp" podStartSLOduration=1.9839869540000001 podStartE2EDuration="4.555473007s" totalImagesPullingTime="2.571486053s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:34.55443442 +0000 UTC m=+335.961014630" watchObservedRunningTime="2026-09-16 14:13:34.555473007 +0000 UTC m=+335.962053215"
Sep 16 14:13:47.838709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.838676    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"]
Sep 16 14:13:47.842122 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.842104    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:47.845208 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.845184    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 16 14:13:47.845332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.845184    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ll6mg\""
Sep 16 14:13:47.845332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.845187    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 16 14:13:47.845332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.845192    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 16 14:13:47.857012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.856989    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"]
Sep 16 14:13:47.917310 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.917279    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gvf2\" (UniqueName: \"kubernetes.io/projected/ce6111d7-479b-4c25-b1ff-c736d3a4c766-kube-api-access-2gvf2\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:47.917470 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.917317    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-metrics-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:47.917470 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.917343    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:47.917470 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:47.917362    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce6111d7-479b-4c25-b1ff-c736d3a4c766-manager-config\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.018235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.018202    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gvf2\" (UniqueName: \"kubernetes.io/projected/ce6111d7-479b-4c25-b1ff-c736d3a4c766-kube-api-access-2gvf2\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.018235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.018240    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-metrics-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.018412 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.018265    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.018412 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.018284    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce6111d7-479b-4c25-b1ff-c736d3a4c766-manager-config\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.018987 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.018964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/ce6111d7-479b-4c25-b1ff-c736d3a4c766-manager-config\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.020710 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.020693    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.020798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.020747    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/ce6111d7-479b-4c25-b1ff-c736d3a4c766-metrics-cert\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.027286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.027261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gvf2\" (UniqueName: \"kubernetes.io/projected/ce6111d7-479b-4c25-b1ff-c736d3a4c766-kube-api-access-2gvf2\") pod \"lws-controller-manager-8544d9d757-nm5t2\" (UID: \"ce6111d7-479b-4c25-b1ff-c736d3a4c766\") " pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.151144 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.151057    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:48.423351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.423278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"]
Sep 16 14:13:48.427084 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:13:48.427056    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce6111d7_479b_4c25_b1ff_c736d3a4c766.slice/crio-e6060a3f641cf5f18e8e12ec4fb5098255e9c3c3a820ced3951dcb6e7facf3ba WatchSource:0}: Error finding container e6060a3f641cf5f18e8e12ec4fb5098255e9c3c3a820ced3951dcb6e7facf3ba: Status 404 returned error can't find the container with id e6060a3f641cf5f18e8e12ec4fb5098255e9c3c3a820ced3951dcb6e7facf3ba
Sep 16 14:13:48.592868 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:48.592834    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2" event={"ID":"ce6111d7-479b-4c25-b1ff-c736d3a4c766","Type":"ContainerStarted","Data":"e6060a3f641cf5f18e8e12ec4fb5098255e9c3c3a820ced3951dcb6e7facf3ba"}
Sep 16 14:13:50.602670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:50.602637    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2" event={"ID":"ce6111d7-479b-4c25-b1ff-c736d3a4c766","Type":"ContainerStarted","Data":"e9ddc55ae25a42f7a8d3ba315f218f6d4dfa3e2c4d3d1b6c26351931ff6df2c8"}
Sep 16 14:13:50.603029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:50.602771    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:13:50.620187 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:50.620144    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2" podStartSLOduration=1.908545913 podStartE2EDuration="3.620131383s" totalImagesPullingTime="1.71158547s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:13:50.619159646 +0000 UTC m=+352.025739853" watchObservedRunningTime="2026-09-16 14:13:50.620131383 +0000 UTC m=+352.026711644"
Sep 16 14:13:54.286634 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.286550    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"]
Sep 16 14:13:54.289765 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.289748    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.295660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.295386    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 16 14:13:54.295660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.295485    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 16 14:13:54.295660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.295497    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 16 14:13:54.295660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.295391    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 16 14:13:54.299122 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.299105    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-plblj\""
Sep 16 14:13:54.326341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.326311    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"]
Sep 16 14:13:54.375991 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.375961    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.375991 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.375991    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.376214 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.376028    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h97d4\" (UniqueName: \"kubernetes.io/projected/e9940bca-6175-4bcb-9a3b-874acf183af3-kube-api-access-h97d4\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.376214 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.376102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.376214 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.376150    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.476630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.476583    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h97d4\" (UniqueName: \"kubernetes.io/projected/e9940bca-6175-4bcb-9a3b-874acf183af3-kube-api-access-h97d4\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.476849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.476647    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.476849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.476668    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.476849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.476718    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.477013 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.476908    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.477102 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.477085    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-manifests\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.479271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.479243    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e9940bca-6175-4bcb-9a3b-874acf183af3-tmp\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.479389 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.479280    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-apiservice-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.479389 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.479370    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e9940bca-6175-4bcb-9a3b-874acf183af3-webhook-cert\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.496780 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.496758    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h97d4\" (UniqueName: \"kubernetes.io/projected/e9940bca-6175-4bcb-9a3b-874acf183af3-kube-api-access-h97d4\") pod \"opendatahub-operator-controller-manager-58d8b99777-w22dd\" (UID: \"e9940bca-6175-4bcb-9a3b-874acf183af3\") " pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.599266 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.599170    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:13:54.881077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:54.880982    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"]
Sep 16 14:13:54.884313 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:13:54.884285    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-ed06561ac4142c68bab4f7f8ebbea15271c52e2bcc913461c5d7bc1e1f07ae64 WatchSource:0}: Error finding container ed06561ac4142c68bab4f7f8ebbea15271c52e2bcc913461c5d7bc1e1f07ae64: Status 404 returned error can't find the container with id ed06561ac4142c68bab4f7f8ebbea15271c52e2bcc913461c5d7bc1e1f07ae64
Sep 16 14:13:55.624645 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:55.624597    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd" event={"ID":"e9940bca-6175-4bcb-9a3b-874acf183af3","Type":"ContainerStarted","Data":"ed06561ac4142c68bab4f7f8ebbea15271c52e2bcc913461c5d7bc1e1f07ae64"}
Sep 16 14:13:58.641013 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:58.640974    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd" event={"ID":"e9940bca-6175-4bcb-9a3b-874acf183af3","Type":"ContainerStarted","Data":"c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f"}
Sep 16 14:13:59.646521 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:59.646479    2580 generic.go:356] "Generic (PLEG): container finished" podID="e9940bca-6175-4bcb-9a3b-874acf183af3" containerID="c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f" exitCode=0
Sep 16 14:13:59.646924 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:13:59.646593    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd" event={"ID":"e9940bca-6175-4bcb-9a3b-874acf183af3","Type":"ContainerDied","Data":"c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f"}
Sep 16 14:13:59.909767 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:13:59.909678    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:00.669519 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:00.669484    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:00.669931 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:00.669681    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:01.608072 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:01.608039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-8544d9d757-nm5t2"
Sep 16 14:14:05.476211 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:05.476177    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:05.673294 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:05.673252    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd" event={"ID":"e9940bca-6175-4bcb-9a3b-874acf183af3","Type":"ContainerStarted","Data":"a771e17a010b4eb02c51e79434ccc9c69cee8c2572ce7448497149e4a4e21b3e"}
Sep 16 14:14:05.673474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:05.673328    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:14:05.697750 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:05.697694    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd" podStartSLOduration=1.775499943 podStartE2EDuration="11.697678091s" totalImagesPullingTime="8.8778483s" totalInitContainerRuntime="1.044329848s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:13:54 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:05.696098912 +0000 UTC m=+367.102679144" watchObservedRunningTime="2026-09-16 14:14:05.697678091 +0000 UTC m=+367.104258301"
Sep 16 14:14:09.917741 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:09.917709    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:16.680762 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:16.680728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-58d8b99777-w22dd"
Sep 16 14:14:19.953343 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:19.953308    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:20.459420 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:20.459387    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:26.963550 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.963518    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"]
Sep 16 14:14:26.967497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.967483    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:26.970484 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.970463    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-zfzt8\""
Sep 16 14:14:26.971088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.971073    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 16 14:14:26.971182 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.971108    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 16 14:14:26.972769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.972749    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 16 14:14:26.982215 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:26.982192    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"]
Sep 16 14:14:27.039765 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039736    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.039925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.039925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039803    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735668c-474c-42f8-9080-8169919886e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.039925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039822    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.039925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039895    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.040148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039932    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.040148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039958    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cj26\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-kube-api-access-5cj26\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.040148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.039989    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735668c-474c-42f8-9080-8169919886e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.040148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.040070    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.140907 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.140869    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.140915    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735668c-474c-42f8-9080-8169919886e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.140944    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.140976    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141000    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141024    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cj26\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-kube-api-access-5cj26\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141063    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735668c-474c-42f8-9080-8169919886e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141170    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141745 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141714    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141870 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141782    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.141939 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141915    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.142010 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.141954    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5735668c-474c-42f8-9080-8169919886e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.143480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.143459    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5735668c-474c-42f8-9080-8169919886e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.143645 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.143627    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5735668c-474c-42f8-9080-8169919886e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.151294 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.151264    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cj26\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-kube-api-access-5cj26\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.151388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.151371    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5735668c-474c-42f8-9080-8169919886e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-68cf4d799cktcsl\" (UID: \"5735668c-474c-42f8-9080-8169919886e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.277235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.277139    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:27.554028 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.553954    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"]
Sep 16 14:14:27.557244 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:14:27.557216    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5735668c_474c_42f8_9080_8169919886e0.slice/crio-4773a29bd96c86dcf5a158fc17578c0cf0e97c3fd3e92f50ef676ea0bb4f269a WatchSource:0}: Error finding container 4773a29bd96c86dcf5a158fc17578c0cf0e97c3fd3e92f50ef676ea0bb4f269a: Status 404 returned error can't find the container with id 4773a29bd96c86dcf5a158fc17578c0cf0e97c3fd3e92f50ef676ea0bb4f269a
Sep 16 14:14:27.772366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:27.772330    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl" event={"ID":"5735668c-474c-42f8-9080-8169919886e0","Type":"ContainerStarted","Data":"4773a29bd96c86dcf5a158fc17578c0cf0e97c3fd3e92f50ef676ea0bb4f269a"}
Sep 16 14:14:29.268363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.268330    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cv5gw"]
Sep 16 14:14:29.272463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.272435    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.277010 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.276987    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 16 14:14:29.282652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.282609    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-rhfgg\""
Sep 16 14:14:29.295109 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.295081    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cv5gw"]
Sep 16 14:14:29.361734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.361688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-47w2f\" (UniqueName: \"kubernetes.io/projected/dce8ca82-336f-4cc2-9a2e-06b058199aab-kube-api-access-47w2f\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.361734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.361730    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.463289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.463252    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-47w2f\" (UniqueName: \"kubernetes.io/projected/dce8ca82-336f-4cc2-9a2e-06b058199aab-kube-api-access-47w2f\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.463289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.463291    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.463522 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:29.463414    2580 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 16 14:14:29.463522 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:29.463480    2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert podName:dce8ca82-336f-4cc2-9a2e-06b058199aab nodeName:}" failed. No retries permitted until 2026-09-16 14:14:29.963460379 +0000 UTC m=+391.370040566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert") pod "odh-model-controller-7998c67dc4-cv5gw" (UID: "dce8ca82-336f-4cc2-9a2e-06b058199aab") : secret "odh-model-controller-webhook-cert" not found
Sep 16 14:14:29.472994 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.472964    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-47w2f\" (UniqueName: \"kubernetes.io/projected/dce8ca82-336f-4cc2-9a2e-06b058199aab-kube-api-access-47w2f\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.968405 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.968365    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:29.971088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:29.971063    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/dce8ca82-336f-4cc2-9a2e-06b058199aab-cert\") pod \"odh-model-controller-7998c67dc4-cv5gw\" (UID: \"dce8ca82-336f-4cc2-9a2e-06b058199aab\") " pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:30.002280 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:30.002238    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:30.185986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.185950    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:30.289527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.289489    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:14:30.289939 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.289593    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:14:30.289939 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.289649    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:14:30.627032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.626941    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-cv5gw"]
Sep 16 14:14:30.631317 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:14:30.631290    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddce8ca82_336f_4cc2_9a2e_06b058199aab.slice/crio-89609a54c724c9c3abfe05cb9aca43480b7b260b61c38dfce4bb0f621dae1ab6 WatchSource:0}: Error finding container 89609a54c724c9c3abfe05cb9aca43480b7b260b61c38dfce4bb0f621dae1ab6: Status 404 returned error can't find the container with id 89609a54c724c9c3abfe05cb9aca43480b7b260b61c38dfce4bb0f621dae1ab6
Sep 16 14:14:30.788553 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.788512    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw" event={"ID":"dce8ca82-336f-4cc2-9a2e-06b058199aab","Type":"ContainerStarted","Data":"89609a54c724c9c3abfe05cb9aca43480b7b260b61c38dfce4bb0f621dae1ab6"}
Sep 16 14:14:30.790087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.790067    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl" event={"ID":"5735668c-474c-42f8-9080-8169919886e0","Type":"ContainerStarted","Data":"7ffcbd957059bd1bd56bfaf2cbd59b6234adf3c9911b3e3feea9070641c5eb3d"}
Sep 16 14:14:30.822324 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:30.822284    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl" podStartSLOduration=2.092313016 podStartE2EDuration="4.822270138s" totalImagesPullingTime="2.729957122s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:30.821956825 +0000 UTC m=+392.228537043" watchObservedRunningTime="2026-09-16 14:14:30.822270138 +0000 UTC m=+392.228850346"
Sep 16 14:14:31.277508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:31.277478    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:31.282800 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:31.282773    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:31.795678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:31.795613    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:31.796785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:31.796757    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-68cf4d799cktcsl"
Sep 16 14:14:33.807413 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:33.807316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw" event={"ID":"dce8ca82-336f-4cc2-9a2e-06b058199aab","Type":"ContainerStarted","Data":"7e7c37f4a3bdd9572cc3c27f10e0ebeb6faee527c4db915f1b578cb8327418c3"}
Sep 16 14:14:33.832661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:33.832611    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw" podStartSLOduration=1.9722625310000002 podStartE2EDuration="4.832594615s" totalImagesPullingTime="2.860332084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:33.83245039 +0000 UTC m=+395.239030598" watchObservedRunningTime="2026-09-16 14:14:33.832594615 +0000 UTC m=+395.239174823"
Sep 16 14:14:34.812082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:34.812048    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:35.490912 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:35.490884    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:36.319255 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.319222    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"]
Sep 16 14:14:36.322403 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.322387    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.325089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.325062    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 16 14:14:36.325222 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.325085    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 16 14:14:36.325222 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.325188    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 16 14:14:36.325718 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.325700    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-bbsvf\""
Sep 16 14:14:36.336523 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.336502    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"]
Sep 16 14:14:36.430390 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430358    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430390 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430393    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430413    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ba357f-a0e0-4c31-8480-c2d35f27e564-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430580    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430603    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxw6z\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-kube-api-access-jxw6z\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.430747 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.430628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.531915 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.531881    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.531915 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.531914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.531935    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ba357f-a0e0-4c31-8480-c2d35f27e564-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.531980    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.532017    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.532040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jxw6z\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-kube-api-access-jxw6z\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.532098    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.532709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.532678    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.534478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.534448    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-kubeconfig\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.534633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.534497    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.534633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.534526    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/c9ba357f-a0e0-4c31-8480-c2d35f27e564-local-certs\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.534633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.534609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/c9ba357f-a0e0-4c31-8480-c2d35f27e564-cacerts\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.550460 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.550438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxw6z\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-kube-api-access-jxw6z\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.550601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.550531    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c9ba357f-a0e0-4c31-8480-c2d35f27e564-istio-token\") pod \"istiod-openshift-gateway-5677ccb49b-q6r7r\" (UID: \"c9ba357f-a0e0-4c31-8480-c2d35f27e564\") " pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.631836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.631769    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:36.947499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:36.947471    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"]
Sep 16 14:14:36.950131 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:14:36.950108    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9ba357f_a0e0_4c31_8480_c2d35f27e564.slice/crio-f202f790c942d0cc260ff3c407e66393d7287c75bf662c08f3ab33770bb108fd WatchSource:0}: Error finding container f202f790c942d0cc260ff3c407e66393d7287c75bf662c08f3ab33770bb108fd: Status 404 returned error can't find the container with id f202f790c942d0cc260ff3c407e66393d7287c75bf662c08f3ab33770bb108fd
Sep 16 14:14:37.832531 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:37.832489    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r" event={"ID":"c9ba357f-a0e0-4c31-8480-c2d35f27e564","Type":"ContainerStarted","Data":"f202f790c942d0cc260ff3c407e66393d7287c75bf662c08f3ab33770bb108fd"}
Sep 16 14:14:39.284293 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:39.284243    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:14:39.284593 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:39.284313    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:14:39.845194 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:39.845157    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r" event={"ID":"c9ba357f-a0e0-4c31-8480-c2d35f27e564","Type":"ContainerStarted","Data":"5f4bc8f4b5dd5cd7dad22fbd55fd9d35f47d3ba57dfaa7fe64a26e1b5cdb11dc"}
Sep 16 14:14:39.845370 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:39.845256    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:39.888058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:39.887991    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r" podStartSLOduration=1.556220362 podStartE2EDuration="3.887966934s" totalImagesPullingTime="2.331746572s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:14:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:14:39.885615503 +0000 UTC m=+401.292195712" watchObservedRunningTime="2026-09-16 14:14:39.887966934 +0000 UTC m=+401.294547144"
Sep 16 14:14:40.012572 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:40.012531    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:40.851186 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:40.851159    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-5677ccb49b-q6r7r"
Sep 16 14:14:45.818971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:14:45.818937    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-cv5gw"
Sep 16 14:14:50.050021 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:50.049983    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:14:50.460242 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:14:50.460205    2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode9940bca_6175_4bcb_9a3b_874acf183af3.slice/crio-c64288fe881704458dc56136502dd532c91e956142a186ac1851547b4450cd0f.scope\": RecentStats: unable to find data in memory cache]"
Sep 16 14:15:59.014144 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:15:59.014112    2580 scope.go:117] "RemoveContainer" containerID="40473dc0d87e35c108ddbb19faf54a4e2854b64e52b175845679e72ed8792db6"
Sep 16 14:16:44.829570 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.829524    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"]
Sep 16 14:16:44.832637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.832620    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.835369 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.835350    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-g587l\""
Sep 16 14:16:44.845264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.845243    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"]
Sep 16 14:16:44.918543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918517    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426402ad-507c-4227-96a5-70718b421c25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918671 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918573    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918671 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918610    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426402ad-507c-4227-96a5-70718b421c25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918671 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918629    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q67jh\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-kube-api-access-q67jh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918672    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918691    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918709    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:44.918899 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:44.918788    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.019790 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019748    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.019944 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019801    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.019944 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.019944 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426402ad-507c-4227-96a5-70718b421c25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.019944 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019927    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019954    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426402ad-507c-4227-96a5-70718b421c25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.019984    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q67jh\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-kube-api-access-q67jh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020081    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020227    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-data\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020223    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020573 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020539    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-credential-socket\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020644    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/426402ad-507c-4227-96a5-70718b421c25-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.020673 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.020656    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-workload-certs\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.022324 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.022308    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/426402ad-507c-4227-96a5-70718b421c25-istio-envoy\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.022518 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.022502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/426402ad-507c-4227-96a5-70718b421c25-istio-podinfo\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.028052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.028028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-istio-token\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.028275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.028257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q67jh\" (UniqueName: \"kubernetes.io/projected/426402ad-507c-4227-96a5-70718b421c25-kube-api-access-q67jh\") pod \"maas-default-gateway-openshift-default-5ccb968c66-zf46t\" (UID: \"426402ad-507c-4227-96a5-70718b421c25\") " pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.141975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.141910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:45.421935 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.421863    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"]
Sep 16 14:16:45.425179 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:16:45.425151    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod426402ad_507c_4227_96a5_70718b421c25.slice/crio-c0c16223a96b5536a72fa57510921cfb9871706c25fd0225681d92765e95d480 WatchSource:0}: Error finding container c0c16223a96b5536a72fa57510921cfb9871706c25fd0225681d92765e95d480: Status 404 returned error can't find the container with id c0c16223a96b5536a72fa57510921cfb9871706c25fd0225681d92765e95d480
Sep 16 14:16:45.427104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.427074    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:16:45.427178 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.427136    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:16:45.427178 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:45.427164    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:16:46.382862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:46.382830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" event={"ID":"426402ad-507c-4227-96a5-70718b421c25","Type":"ContainerStarted","Data":"34de75e004c03f1a53af8d738c1ed40761565f43b36a5f1a3ae99d7ecd61f356"}
Sep 16 14:16:46.383232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:46.382876    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" event={"ID":"426402ad-507c-4227-96a5-70718b421c25","Type":"ContainerStarted","Data":"c0c16223a96b5536a72fa57510921cfb9871706c25fd0225681d92765e95d480"}
Sep 16 14:16:46.405388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:46.405343    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podStartSLOduration=2.405324274 podStartE2EDuration="2.405324274s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:16:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:16:46.404936241 +0000 UTC m=+527.811516452" watchObservedRunningTime="2026-09-16 14:16:46.405324274 +0000 UTC m=+527.811904495"
Sep 16 14:16:47.142814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:47.142775    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:47.147505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:47.147483    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:47.387386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:47.387356    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:16:47.388354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:16:47.388339    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:18:17.151175 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.151142    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:18:17.154577 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.154545    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.173456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.173430    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:18:17.183850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.183826    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mzsgb\""
Sep 16 14:18:17.184953 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.184931    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 14:18:17.312329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.312298    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.312506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.312350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pz6wp\" (UniqueName: \"kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.413400 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.413316    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pz6wp\" (UniqueName: \"kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.413400 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.413396    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.414119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.414101    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.451508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.451474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pz6wp\" (UniqueName: \"kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp\") pod \"payload-processing-5b54b7fbb7-klrpz\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.464315 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.464290    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:17.765185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:17.765147    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:18:17.786697 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:18:17.786335    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod307acd71_5e92_49d4_8267_65052684d4f7.slice/crio-3cc8b9803621182c277f7e6452b3e0b6ff3531cf5cb771fb373e97466892c747 WatchSource:0}: Error finding container 3cc8b9803621182c277f7e6452b3e0b6ff3531cf5cb771fb373e97466892c747: Status 404 returned error can't find the container with id 3cc8b9803621182c277f7e6452b3e0b6ff3531cf5cb771fb373e97466892c747
Sep 16 14:18:18.787787 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:18.787746    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" event={"ID":"307acd71-5e92-49d4-8267-65052684d4f7","Type":"ContainerStarted","Data":"3cc8b9803621182c277f7e6452b3e0b6ff3531cf5cb771fb373e97466892c747"}
Sep 16 14:18:21.808150 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:21.808053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" event={"ID":"307acd71-5e92-49d4-8267-65052684d4f7","Type":"ContainerStarted","Data":"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"}
Sep 16 14:18:21.808547 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:21.808168    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:21.829840 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:21.829784    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" podStartSLOduration=1.063751965 podStartE2EDuration="4.829769685s" totalImagesPullingTime="3.76601772s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:18:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:18:21.827614524 +0000 UTC m=+623.234194733" watchObservedRunningTime="2026-09-16 14:18:21.829769685 +0000 UTC m=+623.236349893"
Sep 16 14:18:26.834669 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.834630    2580 generic.go:356] "Generic (PLEG): container finished" podID="426402ad-507c-4227-96a5-70718b421c25" containerID="34de75e004c03f1a53af8d738c1ed40761565f43b36a5f1a3ae99d7ecd61f356" exitCode=0
Sep 16 14:18:26.835197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.834702    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" event={"ID":"426402ad-507c-4227-96a5-70718b421c25","Type":"ContainerDied","Data":"34de75e004c03f1a53af8d738c1ed40761565f43b36a5f1a3ae99d7ecd61f356"}
Sep 16 14:18:26.835267 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.835201    2580 scope.go:117] "RemoveContainer" containerID="34de75e004c03f1a53af8d738c1ed40761565f43b36a5f1a3ae99d7ecd61f356"
Sep 16 14:18:26.836320 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.836290    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:18:26.836419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.836378    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:18:26.836471 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:26.836455    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:18:27.840682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:27.840650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" event={"ID":"426402ad-507c-4227-96a5-70718b421c25","Type":"ContainerStarted","Data":"bf792514f33e1d13edb658ec7b3438382895306e32579b1d1398c6a1cfe09505"}
Sep 16 14:18:28.142671 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:28.142592    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:18:28.143700 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:28.143678    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zf46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 16 14:18:28.143794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:28.143733    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podUID="426402ad-507c-4227-96a5-70718b421c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 16 14:18:29.142340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:29.142305    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zf46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 16 14:18:29.142774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:29.142361    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podUID="426402ad-507c-4227-96a5-70718b421c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 16 14:18:30.142615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:30.142579    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zf46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 16 14:18:30.142989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:30.142645    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podUID="426402ad-507c-4227-96a5-70718b421c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 16 14:18:31.142359 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:31.142318    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zf46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 16 14:18:31.142534 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:31.142399    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podUID="426402ad-507c-4227-96a5-70718b421c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 16 14:18:32.142627 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:32.142589    2580 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-5ccb968c66-zf46t container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused" start-of-body=
Sep 16 14:18:32.143005 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:32.142649    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t" podUID="426402ad-507c-4227-96a5-70718b421c25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.25:15021/healthz/ready\": dial tcp 10.133.0.25:15021: connect: connection refused"
Sep 16 14:18:32.387929 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:32.387892    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:18:32.815637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:32.815609    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:18:33.376224 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:33.376191    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:18:33.377722 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:18:33.377700    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-5ccb968c66-zf46t"
Sep 16 14:19:09.098086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.098051    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"]
Sep 16 14:19:09.100635 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.100617    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.103362 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.103342    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 16 14:19:09.104156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.104136    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 16 14:19:09.104156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.104146    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 16 14:19:09.104316 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.104139    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-vgn9b\""
Sep 16 14:19:09.111282 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.111262    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"]
Sep 16 14:19:09.172188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172163    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.172188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91456913-976c-4d9f-800d-999605f2f0fc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.172329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172216    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.172329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172236    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88hf2\" (UniqueName: \"kubernetes.io/projected/91456913-976c-4d9f-800d-999605f2f0fc-kube-api-access-88hf2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.172329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172286    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.172426 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.172348    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273176 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273139    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273207    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273237    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91456913-976c-4d9f-800d-999605f2f0fc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273280    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273317    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88hf2\" (UniqueName: \"kubernetes.io/projected/91456913-976c-4d9f-800d-999605f2f0fc-kube-api-access-88hf2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.273387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.273357    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.274062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.274038    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.274496 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.274476    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.274661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.274492    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.278996 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.278900    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/91456913-976c-4d9f-800d-999605f2f0fc-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.285052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.285024    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/91456913-976c-4d9f-800d-999605f2f0fc-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.287011 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.286988    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88hf2\" (UniqueName: \"kubernetes.io/projected/91456913-976c-4d9f-800d-999605f2f0fc-kube-api-access-88hf2\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j\" (UID: \"91456913-976c-4d9f-800d-999605f2f0fc\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.412598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.412522    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:09.686024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.685997    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"]
Sep 16 14:19:09.688639 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:19:09.688610    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91456913_976c_4d9f_800d_999605f2f0fc.slice/crio-d993585f94896bbde568f0484ddc0fde6e9da3d4e416b5d0b5075372bf7d5cec WatchSource:0}: Error finding container d993585f94896bbde568f0484ddc0fde6e9da3d4e416b5d0b5075372bf7d5cec: Status 404 returned error can't find the container with id d993585f94896bbde568f0484ddc0fde6e9da3d4e416b5d0b5075372bf7d5cec
Sep 16 14:19:09.690100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:09.690080    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:19:10.022636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:10.022594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j" event={"ID":"91456913-976c-4d9f-800d-999605f2f0fc","Type":"ContainerStarted","Data":"d993585f94896bbde568f0484ddc0fde6e9da3d4e416b5d0b5075372bf7d5cec"}
Sep 16 14:19:11.033634 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:11.033595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j" event={"ID":"91456913-976c-4d9f-800d-999605f2f0fc","Type":"ContainerStarted","Data":"e7b2d541526466e966f0c7a74a614decd403c4a00220e86a41a8ede79af398e3"}
Sep 16 14:19:11.065970 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:11.065923    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j" podStartSLOduration=1.069655353 podStartE2EDuration="2.06590982s" totalImagesPullingTime="996.254467ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:19:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:19:11.061500088 +0000 UTC m=+672.468080296" watchObservedRunningTime="2026-09-16 14:19:11.06590982 +0000 UTC m=+672.472490027"
Sep 16 14:19:19.413519 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:19.413486    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:19.413981 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:19.413595    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:19.426045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:19.426024    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:19:20.083487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:19:20.083459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-ggp9j"
Sep 16 14:21:01.939770 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:01.939732    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:21:01.940317 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:01.939926    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" podUID="307acd71-5e92-49d4-8267-65052684d4f7" containerName="payload-processing" containerID="cri-o://96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22" gracePeriod=30
Sep 16 14:21:02.186838 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.186814    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:21:02.230963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.230693    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume\") pod \"307acd71-5e92-49d4-8267-65052684d4f7\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") "
Sep 16 14:21:02.230963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.230818    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pz6wp\" (UniqueName: \"kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp\") pod \"307acd71-5e92-49d4-8267-65052684d4f7\" (UID: \"307acd71-5e92-49d4-8267-65052684d4f7\") "
Sep 16 14:21:02.232670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.232294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "307acd71-5e92-49d4-8267-65052684d4f7" (UID: "307acd71-5e92-49d4-8267-65052684d4f7"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:21:02.235035 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.234997    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp" (OuterVolumeSpecName: "kube-api-access-pz6wp") pod "307acd71-5e92-49d4-8267-65052684d4f7" (UID: "307acd71-5e92-49d4-8267-65052684d4f7"). InnerVolumeSpecName "kube-api-access-pz6wp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:21:02.332840 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.332808    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/307acd71-5e92-49d4-8267-65052684d4f7-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:21:02.332840 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.332834    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pz6wp\" (UniqueName: \"kubernetes.io/projected/307acd71-5e92-49d4-8267-65052684d4f7-kube-api-access-pz6wp\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:21:02.510781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.510692    2580 generic.go:356] "Generic (PLEG): container finished" podID="307acd71-5e92-49d4-8267-65052684d4f7" containerID="96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22" exitCode=0
Sep 16 14:21:02.510781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.510743    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" event={"ID":"307acd71-5e92-49d4-8267-65052684d4f7","Type":"ContainerDied","Data":"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"}
Sep 16 14:21:02.510781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.510779    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz" event={"ID":"307acd71-5e92-49d4-8267-65052684d4f7","Type":"ContainerDied","Data":"3cc8b9803621182c277f7e6452b3e0b6ff3531cf5cb771fb373e97466892c747"}
Sep 16 14:21:02.510997 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.510800    2580 scope.go:117] "RemoveContainer" containerID="96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"
Sep 16 14:21:02.510997 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.510743    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-klrpz"
Sep 16 14:21:02.519076 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.519054    2580 scope.go:117] "RemoveContainer" containerID="96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"
Sep 16 14:21:02.519347 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:21:02.519328    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22\": container with ID starting with 96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22 not found: ID does not exist" containerID="96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"
Sep 16 14:21:02.519422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.519354    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22"} err="failed to get container status \"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22\": rpc error: code = NotFound desc = could not find container \"96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22\": container with ID starting with 96e01fe4afb48b2f402f4562f0bf691ed61ff8794174535e84aed40f5e399b22 not found: ID does not exist"
Sep 16 14:21:02.544911 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.544880    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:21:02.552631 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:02.552599    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-klrpz"]
Sep 16 14:21:03.183295 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:03.183252    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="307acd71-5e92-49d4-8267-65052684d4f7" path="/var/lib/kubelet/pods/307acd71-5e92-49d4-8267-65052684d4f7/volumes"
Sep 16 14:21:16.335193 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.335162    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:16.335683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.335507    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="307acd71-5e92-49d4-8267-65052684d4f7" containerName="payload-processing"
Sep 16 14:21:16.335683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.335518    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="307acd71-5e92-49d4-8267-65052684d4f7" containerName="payload-processing"
Sep 16 14:21:16.335683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.335596    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="307acd71-5e92-49d4-8267-65052684d4f7" containerName="payload-processing"
Sep 16 14:21:16.339831 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.339814    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:16.342240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.342219    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"ai-gateway-controller-dockercfg-5kbmp\""
Sep 16 14:21:16.342652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.342635    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"ai-gateway-controller-parameters\""
Sep 16 14:21:16.350809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.350787    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:16.464001 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.463970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5k68v\" (UniqueName: \"kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v\") pod \"ai-gateway-controller-7d7dbcd545-x4ktc\" (UID: \"4751a410-afe5-405c-af59-f53bca05e97f\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:16.564688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.564654    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5k68v\" (UniqueName: \"kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v\") pod \"ai-gateway-controller-7d7dbcd545-x4ktc\" (UID: \"4751a410-afe5-405c-af59-f53bca05e97f\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:16.574192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.574168    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5k68v\" (UniqueName: \"kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v\") pod \"ai-gateway-controller-7d7dbcd545-x4ktc\" (UID: \"4751a410-afe5-405c-af59-f53bca05e97f\") " pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:16.649814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.649731    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:16.926921 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:16.926846    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:16.930056 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:21:16.930029    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4751a410_afe5_405c_af59_f53bca05e97f.slice/crio-6e239289e0d82d988842830cc1b00417d3a23fa251ef67898eef954eae70114b WatchSource:0}: Error finding container 6e239289e0d82d988842830cc1b00417d3a23fa251ef67898eef954eae70114b: Status 404 returned error can't find the container with id 6e239289e0d82d988842830cc1b00417d3a23fa251ef67898eef954eae70114b
Sep 16 14:21:17.580657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:17.580615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" event={"ID":"4751a410-afe5-405c-af59-f53bca05e97f","Type":"ContainerStarted","Data":"6e239289e0d82d988842830cc1b00417d3a23fa251ef67898eef954eae70114b"}
Sep 16 14:21:21.184423 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.184352    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-n28jg"]
Sep 16 14:21:21.187690 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.187670    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.193001 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.192978    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-mzsgb\""
Sep 16 14:21:21.193522 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.193507    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 16 14:21:21.206684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.206661    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-n28jg"]
Sep 16 14:21:21.311196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.311156    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mlcbf\" (UniqueName: \"kubernetes.io/projected/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-kube-api-access-mlcbf\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.311371 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.311248    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.412377 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.412295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mlcbf\" (UniqueName: \"kubernetes.io/projected/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-kube-api-access-mlcbf\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.412377 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.412368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.413536 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.413517    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-plugins-config-volume\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.424910 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.424881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mlcbf\" (UniqueName: \"kubernetes.io/projected/0fa89d75-5a6e-461f-8be3-3d5018ae82dc-kube-api-access-mlcbf\") pod \"payload-processing-5d7dbbd56b-n28jg\" (UID: \"0fa89d75-5a6e-461f-8be3-3d5018ae82dc\") " pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.497845 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.497809    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:21.601742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.601703    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" event={"ID":"4751a410-afe5-405c-af59-f53bca05e97f","Type":"ContainerStarted","Data":"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"}
Sep 16 14:21:21.601926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.601793    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:21.625616 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.625569    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" podStartSLOduration=1.821177725 podStartE2EDuration="5.625538144s" totalImagesPullingTime="3.804360419s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:16 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:21.624740726 +0000 UTC m=+803.031320939" watchObservedRunningTime="2026-09-16 14:21:21.625538144 +0000 UTC m=+803.032118352"
Sep 16 14:21:21.776650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:21.776140    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5d7dbbd56b-n28jg"]
Sep 16 14:21:22.608069 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:22.608029    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" event={"ID":"0fa89d75-5a6e-461f-8be3-3d5018ae82dc","Type":"ContainerStarted","Data":"548b8a2088131aa71ac2fa3054f325d7d51d218abce9175b940c76eac9233389"}
Sep 16 14:21:23.614445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:23.614345    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" event={"ID":"0fa89d75-5a6e-461f-8be3-3d5018ae82dc","Type":"ContainerStarted","Data":"8c4ca857d575a26ee49c5ef0596c126d66f6ea416f1c717c9d94b2eae82ec1f6"}
Sep 16 14:21:23.614895 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:23.614463    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:23.635484 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:23.635436    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" podStartSLOduration=1.199161678 podStartE2EDuration="2.63542292s" totalImagesPullingTime="1.436261242s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:23.634119981 +0000 UTC m=+805.040700189" watchObservedRunningTime="2026-09-16 14:21:23.63542292 +0000 UTC m=+805.042003128"
Sep 16 14:21:24.618849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:24.618815    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:21:32.610420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:32.610388    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:32.862775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:32.862699    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"]
Sep 16 14:21:32.868302 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:32.868281    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:32.881572 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:32.881527    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"]
Sep 16 14:21:32.916899 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:32.916860    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghh7s\" (UniqueName: \"kubernetes.io/projected/56ede454-afb2-4be2-96bd-e154a6c37fd3-kube-api-access-ghh7s\") pod \"ai-gateway-controller-d8854dcc5-6kvrk\" (UID: \"56ede454-afb2-4be2-96bd-e154a6c37fd3\") " pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:33.018324 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:33.018287    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghh7s\" (UniqueName: \"kubernetes.io/projected/56ede454-afb2-4be2-96bd-e154a6c37fd3-kube-api-access-ghh7s\") pod \"ai-gateway-controller-d8854dcc5-6kvrk\" (UID: \"56ede454-afb2-4be2-96bd-e154a6c37fd3\") " pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:33.027630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:33.027602    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghh7s\" (UniqueName: \"kubernetes.io/projected/56ede454-afb2-4be2-96bd-e154a6c37fd3-kube-api-access-ghh7s\") pod \"ai-gateway-controller-d8854dcc5-6kvrk\" (UID: \"56ede454-afb2-4be2-96bd-e154a6c37fd3\") " pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:33.178545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:33.178450    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:33.470249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:33.470218    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"]
Sep 16 14:21:33.473389 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:21:33.473361    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56ede454_afb2_4be2_96bd_e154a6c37fd3.slice/crio-e2893f98cd0fec360ae196945e0ace4a018efa1918f7c18f37a95d70353316d8 WatchSource:0}: Error finding container e2893f98cd0fec360ae196945e0ace4a018efa1918f7c18f37a95d70353316d8: Status 404 returned error can't find the container with id e2893f98cd0fec360ae196945e0ace4a018efa1918f7c18f37a95d70353316d8
Sep 16 14:21:33.660261 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:33.660226    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk" event={"ID":"56ede454-afb2-4be2-96bd-e154a6c37fd3","Type":"ContainerStarted","Data":"e2893f98cd0fec360ae196945e0ace4a018efa1918f7c18f37a95d70353316d8"}
Sep 16 14:21:34.620937 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:34.620904    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg"
Sep 16 14:21:34.621284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:34.621268    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:21:34.674184 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:34.674150    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk" event={"ID":"56ede454-afb2-4be2-96bd-e154a6c37fd3","Type":"ContainerStarted","Data":"6360c57572892ec3f4813ae00447ac98121ce0a21619f6c83f5a371a5ff78be7"}
Sep 16 14:21:34.674578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:34.674274    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:45.681299 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:45.681268    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk"
Sep 16 14:21:45.706790 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:45.706741    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/ai-gateway-controller-d8854dcc5-6kvrk" podStartSLOduration=13.327458667 podStartE2EDuration="13.706726365s" totalImagesPullingTime="379.267698ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:21:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:21:34.717723952 +0000 UTC m=+816.124304160" watchObservedRunningTime="2026-09-16 14:21:45.706726365 +0000 UTC m=+827.113306604"
Sep 16 14:21:45.775748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:45.775720    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:45.775942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:45.775918    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" podUID="4751a410-afe5-405c-af59-f53bca05e97f" containerName="manager" containerID="cri-o://d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378" gracePeriod=30
Sep 16 14:21:46.020548 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.020521    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:46.130133 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.130098    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5k68v\" (UniqueName: \"kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v\") pod \"4751a410-afe5-405c-af59-f53bca05e97f\" (UID: \"4751a410-afe5-405c-af59-f53bca05e97f\") "
Sep 16 14:21:46.132296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.132270    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v" (OuterVolumeSpecName: "kube-api-access-5k68v") pod "4751a410-afe5-405c-af59-f53bca05e97f" (UID: "4751a410-afe5-405c-af59-f53bca05e97f"). InnerVolumeSpecName "kube-api-access-5k68v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:21:46.231585 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.231522    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5k68v\" (UniqueName: \"kubernetes.io/projected/4751a410-afe5-405c-af59-f53bca05e97f-kube-api-access-5k68v\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:21:46.732754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.732714    2580 generic.go:356] "Generic (PLEG): container finished" podID="4751a410-afe5-405c-af59-f53bca05e97f" containerID="d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378" exitCode=0
Sep 16 14:21:46.732754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.732758    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"
Sep 16 14:21:46.733285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.732790    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" event={"ID":"4751a410-afe5-405c-af59-f53bca05e97f","Type":"ContainerDied","Data":"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"}
Sep 16 14:21:46.733285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.732875    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc" event={"ID":"4751a410-afe5-405c-af59-f53bca05e97f","Type":"ContainerDied","Data":"6e239289e0d82d988842830cc1b00417d3a23fa251ef67898eef954eae70114b"}
Sep 16 14:21:46.733285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.732899    2580 scope.go:117] "RemoveContainer" containerID="d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"
Sep 16 14:21:46.741298 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.741279    2580 scope.go:117] "RemoveContainer" containerID="d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"
Sep 16 14:21:46.741551 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:21:46.741534    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378\": container with ID starting with d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378 not found: ID does not exist" containerID="d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"
Sep 16 14:21:46.741614 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.741574    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378"} err="failed to get container status \"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378\": rpc error: code = NotFound desc = could not find container \"d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378\": container with ID starting with d536a2779a35315d2418f0edfba8562a6dbdd587519ebe5ade5ed9e976a96378 not found: ID does not exist"
Sep 16 14:21:46.765013 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.764991    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:46.772752 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:46.772724    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/ai-gateway-controller-7d7dbcd545-x4ktc"]
Sep 16 14:21:47.184347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:21:47.184269    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4751a410-afe5-405c-af59-f53bca05e97f" path="/var/lib/kubelet/pods/4751a410-afe5-405c-af59-f53bca05e97f/volumes"
Sep 16 14:22:07.862993 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.862959    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:22:07.863602 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.863302    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4751a410-afe5-405c-af59-f53bca05e97f" containerName="manager"
Sep 16 14:22:07.863602 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.863313    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4751a410-afe5-405c-af59-f53bca05e97f" containerName="manager"
Sep 16 14:22:07.863602 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.863399    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4751a410-afe5-405c-af59-f53bca05e97f" containerName="manager"
Sep 16 14:22:07.867902 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.867877    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.873530 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.873508    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-d48266-openshift-default-dockercfg-nl65z\""
Sep 16 14:22:07.889611 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.889568    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:22:07.932976 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.932943    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.932976 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.932976    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.932996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933061    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933128    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933154    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933169    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933339 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933186    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2dx48\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.933339 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.933242    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:07.975409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.975377    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:22:07.978647 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.978625    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:07.982329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.982306    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-11636c-openshift-default-dockercfg-w88tl\""
Sep 16 14:22:07.993882 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:07.993859    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:22:08.034253 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034228    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034253 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034256    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034282    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034345    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034363    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034387    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034440    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034615    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034672    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034699    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034715    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034727    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-945qv\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.034768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034760    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034769    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2dx48\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034823    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034855    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.035071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.034898    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.035325 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.035113    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035374 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.035324    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.035469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.035440    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.036734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.036703    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.036882 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.036869    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.055132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.055106    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.055809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.055791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2dx48\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48\") pod \"e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") " pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.136077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.135983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136026    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136068    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136131    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136158    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136185    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136243    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-945qv\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136487    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136760    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.136912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.136821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.137227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.137204    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.138353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.138331    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.138782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.138766    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.146218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.146198    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-945qv\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.146836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.146817    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token\") pod \"e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") " pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.177478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.177434    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:08.288489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.288457    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:08.461939 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.461916    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:22:08.463893 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:08.463869    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc245481b_2e49_41cf_92ae_b069df013a04.slice/crio-b436d5552f3b2df84f6a8acf78d98217a8011843ba53392e9ff92d220c6a80a7 WatchSource:0}: Error finding container b436d5552f3b2df84f6a8acf78d98217a8011843ba53392e9ff92d220c6a80a7: Status 404 returned error can't find the container with id b436d5552f3b2df84f6a8acf78d98217a8011843ba53392e9ff92d220c6a80a7
Sep 16 14:22:08.466198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.466165    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.466254 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.466232    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.466298 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.466263    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.568330 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.568298    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:22:08.573497 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:08.573034    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e76de17_1ea5_4e05_a7e5_b8b1d9f40b81.slice/crio-dd3c4ea62a95dffdbea659d5a755e0c078d4f3c3f1315a9260a75e13a3c63339 WatchSource:0}: Error finding container dd3c4ea62a95dffdbea659d5a755e0c078d4f3c3f1315a9260a75e13a3c63339: Status 404 returned error can't find the container with id dd3c4ea62a95dffdbea659d5a755e0c078d4f3c3f1315a9260a75e13a3c63339
Sep 16 14:22:08.575134 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.575093    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.575235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.575186    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.575235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.575224    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:08.835221 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.835052    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerStarted","Data":"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"}
Sep 16 14:22:08.835221 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.835131    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerStarted","Data":"dd3c4ea62a95dffdbea659d5a755e0c078d4f3c3f1315a9260a75e13a3c63339"}
Sep 16 14:22:08.837123 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.837089    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerStarted","Data":"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"}
Sep 16 14:22:08.837267 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.837146    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerStarted","Data":"b436d5552f3b2df84f6a8acf78d98217a8011843ba53392e9ff92d220c6a80a7"}
Sep 16 14:22:08.872686 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.872627    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podStartSLOduration=1.8726064180000002 podStartE2EDuration="1.872606418s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:08.870497323 +0000 UTC m=+850.277077535" watchObservedRunningTime="2026-09-16 14:22:08.872606418 +0000 UTC m=+850.279186630"
Sep 16 14:22:08.905965 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:08.905917    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podStartSLOduration=1.9059004320000001 podStartE2EDuration="1.905900432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:08.904208038 +0000 UTC m=+850.310788247" watchObservedRunningTime="2026-09-16 14:22:08.905900432 +0000 UTC m=+850.312480639"
Sep 16 14:22:09.181319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.181234    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:09.187748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.187723    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:09.289583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.289530    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:09.295206 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.295175    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:09.843374 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.843335    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:09.843613 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.843396    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:09.844175 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.844152    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:09.844256 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:09.844205    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:13.760078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.760042    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:22:13.763487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.763471    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.768056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.768034    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w3-d48266\""
Sep 16 14:22:13.774088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.774066    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:22:13.777231 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.777215    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:13.782184 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.782164    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w3-d48266-dockercfg-njhxt\""
Sep 16 14:22:13.792065 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.792045    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:22:13.793728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.793700    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmqd5\" (UniqueName: \"kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.793820 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.793764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.805547 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.805521    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:22:13.895258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.895217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmqd5\" (UniqueName: \"kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.895468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.895276    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.895468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.895306    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:13.895468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.895342    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lcm6k\" (UniqueName: \"kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:13.896044 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.896025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.910133 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.910098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmqd5\" (UniqueName: \"kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5\") pod \"payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:13.996259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.996231    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:13.996400 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.996271    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lcm6k\" (UniqueName: \"kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:13.996900 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:13.996878    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:14.009882 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.009856    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lcm6k\" (UniqueName: \"kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k\") pod \"payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") " pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:14.074618 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.074528    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:14.086678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.086657    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:14.362151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.362124    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:22:14.363861 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:14.363837    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod160a6fbc_2f10_4cd5_9140_b81c51b8da99.slice/crio-eca45b4ef20613babde350b1e2222e30efd3244c4047d72fffab9656ec42cdda WatchSource:0}: Error finding container eca45b4ef20613babde350b1e2222e30efd3244c4047d72fffab9656ec42cdda: Status 404 returned error can't find the container with id eca45b4ef20613babde350b1e2222e30efd3244c4047d72fffab9656ec42cdda
Sep 16 14:22:14.386396 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.386373    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:22:14.401692 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:14.401669    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5041bc9f_861b_4457_910e_dcee3780e4f3.slice/crio-579ec7a8a15c8b64bbe4c761f86230b274b6c9276add353dff7db06639cd6354 WatchSource:0}: Error finding container 579ec7a8a15c8b64bbe4c761f86230b274b6c9276add353dff7db06639cd6354: Status 404 returned error can't find the container with id 579ec7a8a15c8b64bbe4c761f86230b274b6c9276add353dff7db06639cd6354
Sep 16 14:22:14.876332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.876294    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" event={"ID":"160a6fbc-2f10-4cd5-9140-b81c51b8da99","Type":"ContainerStarted","Data":"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"}
Sep 16 14:22:14.876848 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.876359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" event={"ID":"160a6fbc-2f10-4cd5-9140-b81c51b8da99","Type":"ContainerStarted","Data":"eca45b4ef20613babde350b1e2222e30efd3244c4047d72fffab9656ec42cdda"}
Sep 16 14:22:14.877081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.877058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:14.878469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.878443    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" event={"ID":"5041bc9f-861b-4457-910e-dcee3780e4f3","Type":"ContainerStarted","Data":"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"}
Sep 16 14:22:14.878606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.878488    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" event={"ID":"5041bc9f-861b-4457-910e-dcee3780e4f3","Type":"ContainerStarted","Data":"579ec7a8a15c8b64bbe4c761f86230b274b6c9276add353dff7db06639cd6354"}
Sep 16 14:22:14.878606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.878511    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:14.941901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:14.941853    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" podStartSLOduration=1.941840287 podStartE2EDuration="1.941840287s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:14.93850295 +0000 UTC m=+856.345083171" watchObservedRunningTime="2026-09-16 14:22:14.941840287 +0000 UTC m=+856.348420495"
Sep 16 14:22:15.013338 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.013289    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" podStartSLOduration=2.013272708 podStartE2EDuration="2.013272708s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:15.011030081 +0000 UTC m=+856.417610291" watchObservedRunningTime="2026-09-16 14:22:15.013272708 +0000 UTC m=+856.419852916"
Sep 16 14:22:15.698816 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.698783    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:22:15.702236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.702217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.714110 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.714091    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-11636c\""
Sep 16 14:22:15.719026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.719001    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-worker-w0-11636c-dockercfg-sps7w\""
Sep 16 14:22:15.738309 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.738282    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:22:15.813729 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.813697    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.813899 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.813775    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sn8pf\" (UniqueName: \"kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.910346 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.910317    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:22:15.913828 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.913805    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:15.914277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.914254    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sn8pf\" (UniqueName: \"kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.914422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.914377    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.915328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.915300    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.953137 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.953061    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sn8pf\" (UniqueName: \"kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf\") pod \"payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:15.980506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:15.980479    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:22:16.013100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.013069    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:16.015510 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.015487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pg8fr\" (UniqueName: \"kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.015649 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.015581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.116427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.116388    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.116618 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.116494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pg8fr\" (UniqueName: \"kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.117284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.117261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.153507 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.153479    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pg8fr\" (UniqueName: \"kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr\") pod \"payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") " pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.224856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.224776    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:16.381840 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.381812    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:22:16.384206 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:16.384174    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3920cc91_0b37_4141_bf1f_fab93c9977b9.slice/crio-d4276cc98ba1ea9753e2200d9efce1597367c2c18a6a4ae7c72edfc7cc6901ea WatchSource:0}: Error finding container d4276cc98ba1ea9753e2200d9efce1597367c2c18a6a4ae7c72edfc7cc6901ea: Status 404 returned error can't find the container with id d4276cc98ba1ea9753e2200d9efce1597367c2c18a6a4ae7c72edfc7cc6901ea
Sep 16 14:22:16.571812 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.571784    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:22:16.572520 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:16.572485    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50cbf182_f238_4b7b_8cd1_4baf4c56952b.slice/crio-994dcef474f264ec3f0d3aaf598685367fe1d0e4057a7040a5797352873b9689 WatchSource:0}: Error finding container 994dcef474f264ec3f0d3aaf598685367fe1d0e4057a7040a5797352873b9689: Status 404 returned error can't find the container with id 994dcef474f264ec3f0d3aaf598685367fe1d0e4057a7040a5797352873b9689
Sep 16 14:22:16.890050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.889944    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" event={"ID":"3920cc91-0b37-4141-bf1f-fab93c9977b9","Type":"ContainerStarted","Data":"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"}
Sep 16 14:22:16.890050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.890005    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" event={"ID":"3920cc91-0b37-4141-bf1f-fab93c9977b9","Type":"ContainerStarted","Data":"d4276cc98ba1ea9753e2200d9efce1597367c2c18a6a4ae7c72edfc7cc6901ea"}
Sep 16 14:22:16.890322 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.890137    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:16.891754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.891726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" event={"ID":"50cbf182-f238-4b7b-8cd1-4baf4c56952b","Type":"ContainerStarted","Data":"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"}
Sep 16 14:22:16.891868 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.891783    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" event={"ID":"50cbf182-f238-4b7b-8cd1-4baf4c56952b","Type":"ContainerStarted","Data":"994dcef474f264ec3f0d3aaf598685367fe1d0e4057a7040a5797352873b9689"}
Sep 16 14:22:16.949209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.949165    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" podStartSLOduration=1.94914901 podStartE2EDuration="1.94914901s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:16.948627293 +0000 UTC m=+858.355207504" watchObservedRunningTime="2026-09-16 14:22:16.94914901 +0000 UTC m=+858.355729209"
Sep 16 14:22:16.975929 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:16.975888    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" podStartSLOduration=1.975873548 podStartE2EDuration="1.975873548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:16.975242453 +0000 UTC m=+858.381822690" watchObservedRunningTime="2026-09-16 14:22:16.975873548 +0000 UTC m=+858.382453757"
Sep 16 14:22:17.904979 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:17.904945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:25.885715 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:25.885669    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 16 14:22:25.886128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:25.885740    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 16 14:22:25.886128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:25.885865    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:22:26.893958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.893933    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:22:26.953484 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.953451    2580 generic.go:356] "Generic (PLEG): container finished" podID="c245481b-2e49-41cf-92ae-b069df013a04" containerID="2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c" exitCode=0
Sep 16 14:22:26.953674 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.953498    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerDied","Data":"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"}
Sep 16 14:22:26.953958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.953942    2580 scope.go:117] "RemoveContainer" containerID="2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"
Sep 16 14:22:26.955017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.954989    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:26.955089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.955056    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:26.955129 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:26.955089    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:27.906732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:27.906701    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:22:27.960778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:27.960748    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerStarted","Data":"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"}
Sep 16 14:22:28.178664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:28.178545    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:28.178849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:28.178692    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:28.178849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:28.178752    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:28.912273 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:28.912238    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:22:29.178012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:29.177915    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:29.178012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:29.177980    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:30.178632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:30.178588    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:30.179119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:30.178661    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:31.186325 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:31.186291    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:31.186923 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:31.186890    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:32.179063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:32.179027    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:32.179359 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:32.179331    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:33.178919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:33.178878    2580 patch_prober.go:28] interesting pod/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused" start-of-body=
Sep 16 14:22:33.179337 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:33.178961    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.31:15021/healthz/ready\": dial tcp 10.133.0.31:15021: connect: connection refused"
Sep 16 14:22:34.182185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:34.182155    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:34.182581 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:34.182413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:34.183113 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:34.183094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:22:38.178845 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:38.178807    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:22:40.399111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.399082    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:22:40.401502 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.401486    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.406380 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.406360    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-b-0adaea-openshift-default-dockercfg-cn6xm\""
Sep 16 14:22:40.446588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.446550    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:22:40.484104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.484062    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.484437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.484412    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.484653 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.484632    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.484795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.484780    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.484961 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.484947    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.485096 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.485082    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.485250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.485235    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.485745 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.485725    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.485942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.485888    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-64gsg\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588066    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588125    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-64gsg\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588264    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588301    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588329    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588418    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.588457    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.590130    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.590647    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.590879    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.591582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.591091    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.592436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.591888    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.594055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.594025    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.594202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.594184    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.610528 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.610508    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-64gsg\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.613722 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.613707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token\") pod \"e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") " pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:40.711539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:40.711492    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:41.018543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:41.018508    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:22:41.019917 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:22:41.019884    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6f7f9cd5_e285_4e50_8929_b78d2ba1ba01.slice/crio-def1dcb8b1d125b1f07277bb48f90b899c0a80ed923262b3e7ecc05cfb59409c WatchSource:0}: Error finding container def1dcb8b1d125b1f07277bb48f90b899c0a80ed923262b3e7ecc05cfb59409c: Status 404 returned error can't find the container with id def1dcb8b1d125b1f07277bb48f90b899c0a80ed923262b3e7ecc05cfb59409c
Sep 16 14:22:41.022106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:41.022077    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:41.022177 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:41.022136    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:41.022177 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:41.022164    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:41.033950 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:41.033921    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerStarted","Data":"def1dcb8b1d125b1f07277bb48f90b899c0a80ed923262b3e7ecc05cfb59409c"}
Sep 16 14:22:42.040113 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:42.040077    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerStarted","Data":"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"}
Sep 16 14:22:42.098292 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:42.098222    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podStartSLOduration=2.098199813 podStartE2EDuration="2.098199813s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:22:42.097135668 +0000 UTC m=+883.503715857" watchObservedRunningTime="2026-09-16 14:22:42.098199813 +0000 UTC m=+883.504780023"
Sep 16 14:22:42.712013 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:42.711980    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:42.717940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:42.717915    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:43.051415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:43.051386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:43.052351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:43.052327    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:22:46.066194 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.066162    2580 generic.go:356] "Generic (PLEG): container finished" podID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerID="42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a" exitCode=0
Sep 16 14:22:46.066621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.066232    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerDied","Data":"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"}
Sep 16 14:22:46.066668 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.066645    2580 scope.go:117] "RemoveContainer" containerID="42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"
Sep 16 14:22:46.067770 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.067745    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:46.067837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.067810    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:46.067876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:46.067841    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:22:47.072552 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:47.072521    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerStarted","Data":"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"}
Sep 16 14:22:47.289071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:47.289026    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:47.289510 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:47.289477    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:47.289644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:47.289538    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:48.289111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:48.289075    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:48.289732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:48.289699    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:49.291930 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:49.291884    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:49.292392 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:49.291942    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:50.289544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:50.289092    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:50.289544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:50.289168    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:51.289402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:51.289366    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:51.289989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:51.289957    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:52.290046 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:52.289609    2580 patch_prober.go:28] interesting pod/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused" start-of-body=
Sep 16 14:22:52.290046 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:52.289675    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.32:15021/healthz/ready\": dial tcp 10.133.0.32:15021: connect: connection refused"
Sep 16 14:22:53.293111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:53.293080    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:53.293487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:53.293361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:22:53.293983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:22:53.293966    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:23:06.174220 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.174119    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerID="8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb" exitCode=0
Sep 16 14:23:06.174220 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.174191    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerDied","Data":"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"}
Sep 16 14:23:06.174670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.174602    2580 scope.go:117] "RemoveContainer" containerID="8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"
Sep 16 14:23:06.175685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.175652    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:23:06.175754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.175736    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:23:06.175790 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:06.175769    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:23:07.195484 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:07.195438    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerStarted","Data":"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"}
Sep 16 14:23:07.712274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:07.712238    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:23:07.712478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:07.712455    2580 patch_prober.go:28] interesting pod/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 16 14:23:07.712524 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:07.712506    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 16 14:23:08.712711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:08.712675    2580 patch_prober.go:28] interesting pod/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 16 14:23:08.713119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:08.712759    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 16 14:23:09.712674 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:09.712639    2580 patch_prober.go:28] interesting pod/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 16 14:23:09.712880 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:09.712704    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 16 14:23:10.711912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:10.711878    2580 patch_prober.go:28] interesting pod/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 16 14:23:10.712092 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:10.711931    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 16 14:23:11.712870 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:11.712835    2580 patch_prober.go:28] interesting pod/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body=
Sep 16 14:23:11.713274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:11.712890    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused"
Sep 16 14:23:12.715698 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:12.715667    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:23:12.716138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:12.715967    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:23:12.716794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:23:12.716777    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:24:00.178497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:24:00.178467    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:25:07.699647 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.699612    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:25:07.702342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.702320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.705404 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.705383    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"distinct-w3-d48266-kserve-self-signed-certs\""
Sep 16 14:25:07.705537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.705442    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"default-dockercfg-fdz6n\""
Sep 16 14:25:07.705537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.705448    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"kube-root-ca.crt\""
Sep 16 14:25:07.705537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.705495    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"openshift-service-ca.crt\""
Sep 16 14:25:07.714681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.714660    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:25:07.782491 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782464    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.782652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782534    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.782652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782586    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.782652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782608    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.782652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.782652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.782646    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wsgn9\" (UniqueName: \"kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.886783 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.886752    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.887161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887093    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887166    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887205    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887233    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887262    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wsgn9\" (UniqueName: \"kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.887813    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.888364    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.889032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.888670    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.890947 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.890907    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.891066 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.891045    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:07.898247 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:07.898208    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wsgn9\" (UniqueName: \"kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9\") pod \"distinct-w3-d48266-kserve-566d849dbb-8l49w\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:08.011204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:08.011134    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:08.293279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:08.287620    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:25:08.784650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:08.784617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" event={"ID":"30bd08aa-129f-4902-9596-3cf89ecec4b7","Type":"ContainerStarted","Data":"60c9575752d5b926e5fac3e4109c7198241c6fe8b5b4274f765f30968726cadb"}
Sep 16 14:25:08.785012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:08.784659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" event={"ID":"30bd08aa-129f-4902-9596-3cf89ecec4b7","Type":"ContainerStarted","Data":"bf7a008e99f079cde7278cd096506fa5711ef0085b37827d3acf87303f5c2ab9"}
Sep 16 14:25:08.810063 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:08.810016    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" podStartSLOduration=1.810000622 podStartE2EDuration="1.810000622s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:25:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:25:08.807345207 +0000 UTC m=+1030.213925616" watchObservedRunningTime="2026-09-16 14:25:08.810000622 +0000 UTC m=+1030.216580835"
Sep 16 14:25:14.178030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:14.177993    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:25:18.011932 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:18.011902    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:18.012303 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:18.011958    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:18.024114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:18.024092    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:18.845279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:18.845248    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:25:43.904644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:43.904598    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:25:43.909850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:43.909827    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:43.912445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:43.912417    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-d48266\"/\"distinct-2-w3-d48266-kserve-self-signed-certs\""
Sep 16 14:25:43.919075 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:43.919053    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:25:44.044251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044219    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.044251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044264    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.044463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.044463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044331    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.044463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044411    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zjkvs\" (UniqueName: \"kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.044600 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.044474    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146071    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146116    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146144    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146163    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146187    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zjkvs\" (UniqueName: \"kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146458 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146368    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146592 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146541    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146589    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.146763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.146748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.148462 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.148446    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.148760 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.148743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.156527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.156469    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zjkvs\" (UniqueName: \"kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs\") pod \"distinct-2-w3-d48266-kserve-7745cbccf7-8xd75\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") " pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.220682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.220651    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:44.495414 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.495391    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:25:44.497453 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:25:44.497427    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0b68a47a_912e_43a1_a73a_809ee23417c0.slice/crio-3cef7c35f980752d548b99d0a1225735e74e98e081f9f7c7c42e4bb497a331cc WatchSource:0}: Error finding container 3cef7c35f980752d548b99d0a1225735e74e98e081f9f7c7c42e4bb497a331cc: Status 404 returned error can't find the container with id 3cef7c35f980752d548b99d0a1225735e74e98e081f9f7c7c42e4bb497a331cc
Sep 16 14:25:44.963830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.963801    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" event={"ID":"0b68a47a-912e-43a1-a73a-809ee23417c0","Type":"ContainerStarted","Data":"150bbd7f2b42ab60cfdb7c4f6b6d2ba2df8c143418580ced1ea479f245fd089c"}
Sep 16 14:25:44.964224 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.963839    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" event={"ID":"0b68a47a-912e-43a1-a73a-809ee23417c0","Type":"ContainerStarted","Data":"3cef7c35f980752d548b99d0a1225735e74e98e081f9f7c7c42e4bb497a331cc"}
Sep 16 14:25:44.987367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:44.987325    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" podStartSLOduration=1.987310473 podStartE2EDuration="1.987310473s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:25:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:25:44.987150864 +0000 UTC m=+1066.393731087" watchObservedRunningTime="2026-09-16 14:25:44.987310473 +0000 UTC m=+1066.393890680"
Sep 16 14:25:54.221663 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:54.221532    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:54.221663 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:54.221627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:54.234196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:54.234169    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:25:55.023087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:25:55.023051    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:26:16.526105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.526074    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:26:16.530016 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.529992    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.532393 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.532372    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-286f09d7-openshift-default-dockercfg-sk59g\""
Sep 16 14:26:16.544200 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.544175    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:26:16.683172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683137    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683185    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683231    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683272    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683310    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683337    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683395    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683425    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnfw4\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.683588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.683466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784006    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784079    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784229    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784480    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784751 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784668    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784751 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784677    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnfw4\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784751 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784707    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.784909 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.784777    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.785285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.785262    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.786419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.786400    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.786742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.786722    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.791632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.791613    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnfw4\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.791837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.791821    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token\") pod \"e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") " pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:16.840065 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:16.840032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:17.123046 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:17.123018    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:26:17.124027 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:26:17.123996    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd192003e_0977_4e9c_91b5_c0e91f069d19.slice/crio-5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74 WatchSource:0}: Error finding container 5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74: Status 404 returned error can't find the container with id 5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74
Sep 16 14:26:17.126398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:17.126365    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:17.126480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:17.126429    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:17.126480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:17.126456    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:18.135064 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.135023    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerStarted","Data":"fbd2532e90b2ab81e84136d68c1cedcc30bd132009fa0550b270dda3d62ea257"}
Sep 16 14:26:18.135468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.135076    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerStarted","Data":"5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74"}
Sep 16 14:26:18.155450 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.155370    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podStartSLOduration=2.155351764 podStartE2EDuration="2.155351764s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:18.154058515 +0000 UTC m=+1099.560638734" watchObservedRunningTime="2026-09-16 14:26:18.155351764 +0000 UTC m=+1099.561931971"
Sep 16 14:26:18.178699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.178664    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:26:18.840626 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.840588    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:18.847078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:18.847039    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:19.140760 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:19.140676    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:19.141670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:19.141650    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:23.435089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.435050    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:26:23.438664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.438647    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.440755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.440727    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-286f09d7\""
Sep 16 14:26:23.440900 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.440882    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-286f09d7-dockercfg-q4776\""
Sep 16 14:26:23.449834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.449803    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:26:23.482505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.482469    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:26:23.486383 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.486363    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.493449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.493426    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:26:23.558449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.558408    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lgs6k\" (UniqueName: \"kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.558781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.558762    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.558946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.558928    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.559181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.559161    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjxqx\" (UniqueName: \"kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.660867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.660834    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjxqx\" (UniqueName: \"kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.661027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.660914    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lgs6k\" (UniqueName: \"kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.661027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.660943    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.661027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.660959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.661638 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.661619    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.661681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.661660    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.668706 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.668684    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjxqx\" (UniqueName: \"kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx\") pod \"payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") " pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.669279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.669257    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lgs6k\" (UniqueName: \"kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k\") pod \"payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:23.751095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.751060    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:23.797597 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:23.797553    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:24.037045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.037016    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:26:24.038438 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:26:24.038412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb4d4368_9cde_4d5e_90c3_592ca89d8927.slice/crio-2154074caebe66ec898669168c1c09709362c024d7bac37110fbba342622d99e WatchSource:0}: Error finding container 2154074caebe66ec898669168c1c09709362c024d7bac37110fbba342622d99e: Status 404 returned error can't find the container with id 2154074caebe66ec898669168c1c09709362c024d7bac37110fbba342622d99e
Sep 16 14:26:24.085334 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.085301    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:26:24.092414 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:26:24.092390    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fb67b5a_a2a6_4e5a_a67a_23c1e82a9d94.slice/crio-344e5beb7106d094749a7b4703ef9cd2d41370b3739eaba23881d573db7c0b57 WatchSource:0}: Error finding container 344e5beb7106d094749a7b4703ef9cd2d41370b3739eaba23881d573db7c0b57: Status 404 returned error can't find the container with id 344e5beb7106d094749a7b4703ef9cd2d41370b3739eaba23881d573db7c0b57
Sep 16 14:26:24.171639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.171337    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" event={"ID":"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94","Type":"ContainerStarted","Data":"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"}
Sep 16 14:26:24.171639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.171408    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" event={"ID":"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94","Type":"ContainerStarted","Data":"344e5beb7106d094749a7b4703ef9cd2d41370b3739eaba23881d573db7c0b57"}
Sep 16 14:26:24.171639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.171440    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:24.173182 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.173155    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" event={"ID":"eb4d4368-9cde-4d5e-90c3-592ca89d8927","Type":"ContainerStarted","Data":"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"}
Sep 16 14:26:24.173301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.173200    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" event={"ID":"eb4d4368-9cde-4d5e-90c3-592ca89d8927","Type":"ContainerStarted","Data":"2154074caebe66ec898669168c1c09709362c024d7bac37110fbba342622d99e"}
Sep 16 14:26:24.173301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.173276    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:24.190694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.190639    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" podStartSLOduration=1.190620445 podStartE2EDuration="1.190620445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:24.187824878 +0000 UTC m=+1105.594405078" watchObservedRunningTime="2026-09-16 14:26:24.190620445 +0000 UTC m=+1105.597200654"
Sep 16 14:26:24.207047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:24.206995    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" podStartSLOduration=1.206980997 podStartE2EDuration="1.206980997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:26:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:26:24.20530004 +0000 UTC m=+1105.611880238" watchObservedRunningTime="2026-09-16 14:26:24.206980997 +0000 UTC m=+1105.613561253"
Sep 16 14:26:35.196855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:35.196820    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:26:35.197334 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:35.196889    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:26:42.279490 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.279408    2580 generic.go:356] "Generic (PLEG): container finished" podID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerID="fbd2532e90b2ab81e84136d68c1cedcc30bd132009fa0550b270dda3d62ea257" exitCode=0
Sep 16 14:26:42.279490 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.279449    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerDied","Data":"fbd2532e90b2ab81e84136d68c1cedcc30bd132009fa0550b270dda3d62ea257"}
Sep 16 14:26:42.279953 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.279835    2580 scope.go:117] "RemoveContainer" containerID="fbd2532e90b2ab81e84136d68c1cedcc30bd132009fa0550b270dda3d62ea257"
Sep 16 14:26:42.280971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.280944    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:42.281032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.281014    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:42.281074 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:42.281044    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:26:43.291001 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:43.290952    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerStarted","Data":"b170e521728ba64afdfdaee230d965d611420ee0d47b5097ff26057c225e3b51"}
Sep 16 14:26:43.840831 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:43.840787    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:43.841032 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:43.840985    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 16 14:26:43.841094 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:43.841033    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 16 14:26:44.841503 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:44.841461    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 16 14:26:44.841916 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:44.841528    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 16 14:26:45.841305 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:45.841267    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 16 14:26:45.841532 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:45.841330    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 16 14:26:46.840438 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:46.840402    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 16 14:26:46.840711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:46.840458    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 16 14:26:47.840631 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:47.840595    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused" start-of-body=
Sep 16 14:26:47.841204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:47.840653    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.40:15021/healthz/ready\": dial tcp 10.133.0.40:15021: connect: connection refused"
Sep 16 14:26:48.845401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:48.845370    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:48.845802 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:48.845657    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:26:48.846429 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:26:48.846405    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:27:01.690922 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:01.690889    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:27:01.691321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:01.691158    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" podUID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" containerName="payload-processing" containerID="cri-o://f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d" gracePeriod=30
Sep 16 14:27:01.707831 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:01.707717    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:27:01.707934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:01.707902    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" podUID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" containerName="payload-pre-processing" containerID="cri-o://b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c" gracePeriod=30
Sep 16 14:27:02.016337 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.016312    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:27:02.020884 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.020857    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:27:02.070959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.070916    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vjxqx\" (UniqueName: \"kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx\") pod \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") "
Sep 16 14:27:02.071151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071085    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume\") pod \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\" (UID: \"eb4d4368-9cde-4d5e-90c3-592ca89d8927\") "
Sep 16 14:27:02.071230 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071170    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lgs6k\" (UniqueName: \"kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k\") pod \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") "
Sep 16 14:27:02.071230 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071218    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume\") pod \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\" (UID: \"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94\") "
Sep 16 14:27:02.071847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "eb4d4368-9cde-4d5e-90c3-592ca89d8927" (UID: "eb4d4368-9cde-4d5e-90c3-592ca89d8927"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:02.071847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071825    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/eb4d4368-9cde-4d5e-90c3-592ca89d8927-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:02.072039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.071846    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" (UID: "8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:02.073918 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.073892    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx" (OuterVolumeSpecName: "kube-api-access-vjxqx") pod "eb4d4368-9cde-4d5e-90c3-592ca89d8927" (UID: "eb4d4368-9cde-4d5e-90c3-592ca89d8927"). InnerVolumeSpecName "kube-api-access-vjxqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:02.074077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.074057    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k" (OuterVolumeSpecName: "kube-api-access-lgs6k") pod "8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" (UID: "8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94"). InnerVolumeSpecName "kube-api-access-lgs6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:02.172650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.172618    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lgs6k\" (UniqueName: \"kubernetes.io/projected/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-kube-api-access-lgs6k\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:02.172650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.172649    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:02.172650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.172662    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vjxqx\" (UniqueName: \"kubernetes.io/projected/eb4d4368-9cde-4d5e-90c3-592ca89d8927-kube-api-access-vjxqx\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:02.198529 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.198495    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:27:02.198745 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.198722    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" containerID="cri-o://b170e521728ba64afdfdaee230d965d611420ee0d47b5097ff26057c225e3b51" gracePeriod=30
Sep 16 14:27:02.393293 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.393208    2580 generic.go:356] "Generic (PLEG): container finished" podID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" containerID="f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d" exitCode=0
Sep 16 14:27:02.393293 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.393252    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"
Sep 16 14:27:02.393487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.393296    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" event={"ID":"eb4d4368-9cde-4d5e-90c3-592ca89d8927","Type":"ContainerDied","Data":"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"}
Sep 16 14:27:02.393487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.393348    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7" event={"ID":"eb4d4368-9cde-4d5e-90c3-592ca89d8927","Type":"ContainerDied","Data":"2154074caebe66ec898669168c1c09709362c024d7bac37110fbba342622d99e"}
Sep 16 14:27:02.393487 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.393375    2580 scope.go:117] "RemoveContainer" containerID="f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"
Sep 16 14:27:02.394700 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.394680    2580 generic.go:356] "Generic (PLEG): container finished" podID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" containerID="b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c" exitCode=0
Sep 16 14:27:02.394794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.394717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"
Sep 16 14:27:02.394794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.394715    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" event={"ID":"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94","Type":"ContainerDied","Data":"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"}
Sep 16 14:27:02.394794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.394772    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn" event={"ID":"8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94","Type":"ContainerDied","Data":"344e5beb7106d094749a7b4703ef9cd2d41370b3739eaba23881d573db7c0b57"}
Sep 16 14:27:02.403846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.403824    2580 scope.go:117] "RemoveContainer" containerID="f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"
Sep 16 14:27:02.404350 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:27:02.404306    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d\": container with ID starting with f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d not found: ID does not exist" containerID="f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"
Sep 16 14:27:02.404463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.404355    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d"} err="failed to get container status \"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d\": rpc error: code = NotFound desc = could not find container \"f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d\": container with ID starting with f06726df48d4ffb7e9e054421fe51379cfc4ebef20f8b416598a73029c409c1d not found: ID does not exist"
Sep 16 14:27:02.404463 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.404372    2580 scope.go:117] "RemoveContainer" containerID="b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"
Sep 16 14:27:02.412443 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.412422    2580 scope.go:117] "RemoveContainer" containerID="b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"
Sep 16 14:27:02.412802 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:27:02.412784    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c\": container with ID starting with b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c not found: ID does not exist" containerID="b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"
Sep 16 14:27:02.412871 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.412810    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c"} err="failed to get container status \"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c\": rpc error: code = NotFound desc = could not find container \"b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c\": container with ID starting with b740e03211328b5c95c3ae863fc49c77ff98d677e983ed4b8dd039a1235a986c not found: ID does not exist"
Sep 16 14:27:02.447031 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.446995    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:27:02.459188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.459158    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-286f09d7-6bb6978cbc-86tb7"]
Sep 16 14:27:02.497644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.497608    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:27:02.499524 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:02.499497    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-286f09d7-556cfd477d-42rtn"]
Sep 16 14:27:03.184973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:03.184889    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" path="/var/lib/kubelet/pods/8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94/volumes"
Sep 16 14:27:03.185353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:03.185246    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" path="/var/lib/kubelet/pods/eb4d4368-9cde-4d5e-90c3-592ca89d8927/volumes"
Sep 16 14:27:03.847685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:03.847650    2580 patch_prober.go:28] interesting pod/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:27:03.847856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:03.847713    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:27:07.438575 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.438526    2580 generic.go:356] "Generic (PLEG): container finished" podID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerID="b170e521728ba64afdfdaee230d965d611420ee0d47b5097ff26057c225e3b51" exitCode=0
Sep 16 14:27:07.438969 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.438601    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerDied","Data":"b170e521728ba64afdfdaee230d965d611420ee0d47b5097ff26057c225e3b51"}
Sep 16 14:27:07.438969 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.438676    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg" event={"ID":"d192003e-0977-4e9c-91b5-c0e91f069d19","Type":"ContainerDied","Data":"5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74"}
Sep 16 14:27:07.438969 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.438708    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5df13a2d153a101461083810fef9e13f98d2ce962b479b568a6ca869ea601c74"
Sep 16 14:27:07.438969 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.438727    2580 scope.go:117] "RemoveContainer" containerID="fbd2532e90b2ab81e84136d68c1cedcc30bd132009fa0550b270dda3d62ea257"
Sep 16 14:27:07.454197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.454176    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:27:07.532410 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532319    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532410 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532398    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532428    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnfw4\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532596    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532619    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:07.532679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532701    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532739    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs\") pod \"d192003e-0977-4e9c-91b5-c0e91f069d19\" (UID: \"d192003e-0977-4e9c-91b5-c0e91f069d19\") "
Sep 16 14:27:07.532975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532816    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data" (OuterVolumeSpecName: "istio-data") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:07.533131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532982    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:07.533131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.532992    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:07.533248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.533151    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.533248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.533172    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d192003e-0977-4e9c-91b5-c0e91f069d19-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.533248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.533187    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.533248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.533199    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.533248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.533198    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:07.534799 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.534779    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token" (OuterVolumeSpecName: "istio-token") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:07.534885 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.534858    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:07.535119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.535103    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:27:07.535165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.535131    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4" (OuterVolumeSpecName: "kube-api-access-vnfw4") pod "d192003e-0977-4e9c-91b5-c0e91f069d19" (UID: "d192003e-0977-4e9c-91b5-c0e91f069d19"). InnerVolumeSpecName "kube-api-access-vnfw4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:07.633834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.633797    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.633834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.633829    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.633834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.633838    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d192003e-0977-4e9c-91b5-c0e91f069d19-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.633834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.633846    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d192003e-0977-4e9c-91b5-c0e91f069d19-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:07.634119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:07.633855    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnfw4\" (UniqueName: \"kubernetes.io/projected/d192003e-0977-4e9c-91b5-c0e91f069d19-kube-api-access-vnfw4\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:08.445967 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:08.445941    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"
Sep 16 14:27:08.495695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:08.495660    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:27:08.501387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:08.501357    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-286f09d7-openshift-default-6c7998bf76-7p6jg"]
Sep 16 14:27:09.185028 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.184995    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" path="/var/lib/kubelet/pods/d192003e-0977-4e9c-91b5-c0e91f069d19/volumes"
Sep 16 14:27:09.826017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.825980    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:09.826415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826400    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" containerName="payload-processing"
Sep 16 14:27:09.826459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826416    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" containerName="payload-processing"
Sep 16 14:27:09.826459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826428    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:09.826459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826434    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:09.826459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826444    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" containerName="payload-pre-processing"
Sep 16 14:27:09.826459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826451    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" containerName="payload-pre-processing"
Sep 16 14:27:09.826646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826528    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="eb4d4368-9cde-4d5e-90c3-592ca89d8927" containerName="payload-processing"
Sep 16 14:27:09.826646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826540    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:09.826646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826549    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:09.826646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.826568    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8fb67b5a-a2a6-4e5a-a67a-23c1e82a9d94" containerName="payload-pre-processing"
Sep 16 14:27:09.831138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.831120    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:09.833309 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.833289    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-beaf471d-dockercfg-gn55x\""
Sep 16 14:27:09.833421 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.833377    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-beaf471d\""
Sep 16 14:27:09.873278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.873241    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:09.956833 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.956794    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m69vt\" (UniqueName: \"kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:09.957037 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:09.956851    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.058030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.057983    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m69vt\" (UniqueName: \"kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.058238 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.058047    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.058814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.058784    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.067844 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.067820    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m69vt\" (UniqueName: \"kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt\") pod \"payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") " pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.140997 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.140915    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:10.433289 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:27:10.433257    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ed8d576_f5f8_4f7c_8bf5_288339736704.slice/crio-27c7a4bc8052d88fd513bf80be711bfe43b7afefe24a73fa6da19a5bae2d0c5b WatchSource:0}: Error finding container 27c7a4bc8052d88fd513bf80be711bfe43b7afefe24a73fa6da19a5bae2d0c5b: Status 404 returned error can't find the container with id 27c7a4bc8052d88fd513bf80be711bfe43b7afefe24a73fa6da19a5bae2d0c5b
Sep 16 14:27:10.435034 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.434953    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:10.459925 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:10.459895    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" event={"ID":"0ed8d576-f5f8-4f7c-8bf5-288339736704","Type":"ContainerStarted","Data":"27c7a4bc8052d88fd513bf80be711bfe43b7afefe24a73fa6da19a5bae2d0c5b"}
Sep 16 14:27:11.467033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:11.466991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" event={"ID":"0ed8d576-f5f8-4f7c-8bf5-288339736704","Type":"ContainerStarted","Data":"3f96837d908b49e2566da5a95f007b1edf5a4e0f004c93c7f5c0d745f1b65bd1"}
Sep 16 14:27:11.467468 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:11.467108    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:22.474316 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:22.474230    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:22.519179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:22.519127    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" podStartSLOduration=13.51911096 podStartE2EDuration="13.51911096s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:11.523400649 +0000 UTC m=+1152.929980854" watchObservedRunningTime="2026-09-16 14:27:22.51911096 +0000 UTC m=+1163.925691167"
Sep 16 14:27:30.150078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:30.150044    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:27:30.150503 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:30.150303    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy" containerID="cri-o://699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d" gracePeriod=30
Sep 16 14:27:35.402227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.402197    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:27:35.511508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511475    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-64gsg\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.511741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.511741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.511741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511629    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.511741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.511741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.512004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511759    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.512004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511780    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.512004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.511812    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs\") pod \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\" (UID: \"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01\") "
Sep 16 14:27:35.512169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512081    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data" (OuterVolumeSpecName: "istio-data") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512263 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512243    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.512319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512262    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.512319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.512319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512287    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:35.512472 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.512326    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.514039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.514006    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:27:35.514039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.514014    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:27:35.514039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.514013    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token" (OuterVolumeSpecName: "istio-token") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:35.514039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.514031    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg" (OuterVolumeSpecName: "kube-api-access-64gsg") pod "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" (UID: "6f7f9cd5-e285-4e50-8929-b78d2ba1ba01"). InnerVolumeSpecName "kube-api-access-64gsg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:35.601109 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.601072    2580 generic.go:356] "Generic (PLEG): container finished" podID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerID="699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d" exitCode=0
Sep 16 14:27:35.601285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.601118    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"
Sep 16 14:27:35.601285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.601126    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerDied","Data":"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"}
Sep 16 14:27:35.601285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.601184    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988" event={"ID":"6f7f9cd5-e285-4e50-8929-b78d2ba1ba01","Type":"ContainerDied","Data":"def1dcb8b1d125b1f07277bb48f90b899c0a80ed923262b3e7ecc05cfb59409c"}
Sep 16 14:27:35.601285 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.601208    2580 scope.go:117] "RemoveContainer" containerID="699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"
Sep 16 14:27:35.610459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.610440    2580 scope.go:117] "RemoveContainer" containerID="8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"
Sep 16 14:27:35.612980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.612957    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-64gsg\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-kube-api-access-64gsg\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.612980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.612982    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.613154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.612991    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.613154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.612999    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.613154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.613007    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.613154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.613014    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.613154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.613022    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:35.619070 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.619050    2580 scope.go:117] "RemoveContainer" containerID="699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"
Sep 16 14:27:35.619333 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:27:35.619313    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d\": container with ID starting with 699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d not found: ID does not exist" containerID="699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"
Sep 16 14:27:35.619415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.619345    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d"} err="failed to get container status \"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d\": rpc error: code = NotFound desc = could not find container \"699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d\": container with ID starting with 699ac7c256ccc7e8c32c6f3881c67e0f1be6618055a9ef2d539016174d1cfb9d not found: ID does not exist"
Sep 16 14:27:35.619415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.619368    2580 scope.go:117] "RemoveContainer" containerID="8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"
Sep 16 14:27:35.619655 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:27:35.619627    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb\": container with ID starting with 8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb not found: ID does not exist" containerID="8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"
Sep 16 14:27:35.619751 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.619662    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb"} err="failed to get container status \"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb\": rpc error: code = NotFound desc = could not find container \"8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb\": container with ID starting with 8a714a6f2ee3c46d341734718c334c798cbea76ce526e5ed51e00c8afcf593bb not found: ID does not exist"
Sep 16 14:27:35.634224 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.634168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:27:35.640756 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:35.640739    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-b-0adaea-openshift-default-648cc57f96-j4988"]
Sep 16 14:27:37.184883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:37.184850    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" path="/var/lib/kubelet/pods/6f7f9cd5-e285-4e50-8929-b78d2ba1ba01/volumes"
Sep 16 14:27:38.474753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.474724    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:38.475145 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.474950    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" podUID="0ed8d576-f5f8-4f7c-8bf5-288339736704" containerName="payload-processing" containerID="cri-o://3f96837d908b49e2566da5a95f007b1edf5a4e0f004c93c7f5c0d745f1b65bd1" gracePeriod=30
Sep 16 14:27:38.627211 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.627172    2580 generic.go:356] "Generic (PLEG): container finished" podID="0ed8d576-f5f8-4f7c-8bf5-288339736704" containerID="3f96837d908b49e2566da5a95f007b1edf5a4e0f004c93c7f5c0d745f1b65bd1" exitCode=0
Sep 16 14:27:38.627388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.627243    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" event={"ID":"0ed8d576-f5f8-4f7c-8bf5-288339736704","Type":"ContainerDied","Data":"3f96837d908b49e2566da5a95f007b1edf5a4e0f004c93c7f5c0d745f1b65bd1"}
Sep 16 14:27:38.740677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.740608    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:38.842325 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.842298    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m69vt\" (UniqueName: \"kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt\") pod \"0ed8d576-f5f8-4f7c-8bf5-288339736704\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") "
Sep 16 14:27:38.842514 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.842338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume\") pod \"0ed8d576-f5f8-4f7c-8bf5-288339736704\" (UID: \"0ed8d576-f5f8-4f7c-8bf5-288339736704\") "
Sep 16 14:27:38.842807 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.842783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0ed8d576-f5f8-4f7c-8bf5-288339736704" (UID: "0ed8d576-f5f8-4f7c-8bf5-288339736704"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:27:38.844384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.844360    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt" (OuterVolumeSpecName: "kube-api-access-m69vt") pod "0ed8d576-f5f8-4f7c-8bf5-288339736704" (UID: "0ed8d576-f5f8-4f7c-8bf5-288339736704"). InnerVolumeSpecName "kube-api-access-m69vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:27:38.943843 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.943810    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m69vt\" (UniqueName: \"kubernetes.io/projected/0ed8d576-f5f8-4f7c-8bf5-288339736704-kube-api-access-m69vt\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:38.943843 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:38.943837    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0ed8d576-f5f8-4f7c-8bf5-288339736704-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:27:39.634323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:39.634238    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"
Sep 16 14:27:39.634323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:39.634239    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr" event={"ID":"0ed8d576-f5f8-4f7c-8bf5-288339736704","Type":"ContainerDied","Data":"27c7a4bc8052d88fd513bf80be711bfe43b7afefe24a73fa6da19a5bae2d0c5b"}
Sep 16 14:27:39.634844 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:39.634373    2580 scope.go:117] "RemoveContainer" containerID="3f96837d908b49e2566da5a95f007b1edf5a4e0f004c93c7f5c0d745f1b65bd1"
Sep 16 14:27:39.667267 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:39.667233    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:39.679879 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:39.679851    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-beaf471d-5d98ff8d6b-xp7wr"]
Sep 16 14:27:40.314386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314357    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:27:40.314887 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314867    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314890    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d192003e-0977-4e9c-91b5-c0e91f069d19" containerName="istio-proxy"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314910    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314918    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314934    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0ed8d576-f5f8-4f7c-8bf5-288339736704" containerName="payload-processing"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314943    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0ed8d576-f5f8-4f7c-8bf5-288339736704" containerName="payload-processing"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314955    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.314986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.314963    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.315321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.315078    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.315321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.315091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0ed8d576-f5f8-4f7c-8bf5-288339736704" containerName="payload-processing"
Sep 16 14:27:40.315321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.315105    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6f7f9cd5-e285-4e50-8929-b78d2ba1ba01" containerName="istio-proxy"
Sep 16 14:27:40.319482 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.319463    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.325416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.325395    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-5f53f6be-openshift-default-dockercfg-njxbr\""
Sep 16 14:27:40.340466 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.340443    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:27:40.460996 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.460968    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461173 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461012    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461173 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461130    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461189    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461253    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461291    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461473 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461350    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2rt9k\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461473 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461384    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.461473 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.461418    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562138    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2rt9k\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562175    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562196    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562238    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562269    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562308    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562328    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562402 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562356    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562729 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562517    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.562823 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562800    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.563019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.562991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.563107 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.563001    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.563151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.563123    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.563205 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.563186    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.565089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.565037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.565266 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.565250    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.574025 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.573991    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2rt9k\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.575629 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.575607    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token\") pod \"e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") " pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.630733 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.630698    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:40.913667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.913644    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:27:40.914682 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:27:40.914658    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda51fdc4e_035e_4442_aa96_dc14ec824829.slice/crio-b790bc786d9eb2eab1cbaf662173937d507dba312283ae307c6d859cee2f1f7e WatchSource:0}: Error finding container b790bc786d9eb2eab1cbaf662173937d507dba312283ae307c6d859cee2f1f7e: Status 404 returned error can't find the container with id b790bc786d9eb2eab1cbaf662173937d507dba312283ae307c6d859cee2f1f7e
Sep 16 14:27:40.916985 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.916956    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:27:40.917074 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.917021    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:27:40.917074 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:40.917060    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:27:41.186108 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:41.186010    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0ed8d576-f5f8-4f7c-8bf5-288339736704" path="/var/lib/kubelet/pods/0ed8d576-f5f8-4f7c-8bf5-288339736704/volumes"
Sep 16 14:27:41.647950 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:41.647915    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerStarted","Data":"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"}
Sep 16 14:27:41.648126 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:41.647976    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerStarted","Data":"b790bc786d9eb2eab1cbaf662173937d507dba312283ae307c6d859cee2f1f7e"}
Sep 16 14:27:41.674861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:41.674811    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podStartSLOduration=1.674791245 podStartE2EDuration="1.674791245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:41.672836434 +0000 UTC m=+1183.079416667" watchObservedRunningTime="2026-09-16 14:27:41.674791245 +0000 UTC m=+1183.081371453"
Sep 16 14:27:42.631051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:42.631007    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:42.635974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:42.635947    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:42.653005 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:42.652980    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:42.653873 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:42.653854    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:27:43.178289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:43.178249    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:27:45.517579 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.517526    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:27:45.524667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.524614    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.528210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.528172    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-5f53f6be\""
Sep 16 14:27:45.528506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.528484    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-5f53f6be-dockercfg-djq6m\""
Sep 16 14:27:45.534174 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.534151    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:27:45.619931 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.619894    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.620111 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.619972    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6dwlq\" (UniqueName: \"kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.720822 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.720786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6dwlq\" (UniqueName: \"kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.720994 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.720888    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.721526 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.721507    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.746296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.746261    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6dwlq\" (UniqueName: \"kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq\") pod \"payload-processing-e2e-ait-5f53f6be-688549f475-f66c2\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") " pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:45.839164 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:45.839075    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:46.140042 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:46.140013    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:27:46.142345 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:27:46.142309    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e223c0c_ac75_45ed_8761_ecc83024c147.slice/crio-a0ddba9d4fd90f1f2e97e6edc7428d4a1ffd744bdb1a981053319e476cb873c9 WatchSource:0}: Error finding container a0ddba9d4fd90f1f2e97e6edc7428d4a1ffd744bdb1a981053319e476cb873c9: Status 404 returned error can't find the container with id a0ddba9d4fd90f1f2e97e6edc7428d4a1ffd744bdb1a981053319e476cb873c9
Sep 16 14:27:46.676621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:46.676582    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" event={"ID":"5e223c0c-ac75-45ed-8761-ecc83024c147","Type":"ContainerStarted","Data":"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"}
Sep 16 14:27:46.677015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:46.676638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" event={"ID":"5e223c0c-ac75-45ed-8761-ecc83024c147","Type":"ContainerStarted","Data":"a0ddba9d4fd90f1f2e97e6edc7428d4a1ffd744bdb1a981053319e476cb873c9"}
Sep 16 14:27:46.677015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:46.676674    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:57.683974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:57.683945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:27:57.712149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:27:57.712098    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" podStartSLOduration=12.712082779 podStartE2EDuration="12.712082779s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:27:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:27:46.744822063 +0000 UTC m=+1188.151402271" watchObservedRunningTime="2026-09-16 14:27:57.712082779 +0000 UTC m=+1199.118662986"
Sep 16 14:28:05.318010 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.317978    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:28:05.323156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.323128    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.326066 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.326044    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w2-d31d7c-openshift-default-dockercfg-5nzkd\""
Sep 16 14:28:05.333808 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.333777    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:28:05.422629 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422595    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422638    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422743    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422785    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422852    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wqhd\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422873    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422919    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.422946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.422939    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523514 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523471    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7wqhd\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523514 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523516    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523545    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523710    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523767    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523793    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.523830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523821    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.523969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524193 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.524139    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524193 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.524165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.524335    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524423 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.524369    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.524628 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.524609    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.526641 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.526621    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.526769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.526752    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.536523 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.536502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.541018 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.540996    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wqhd\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd\") pod \"e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") " pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.634999 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.634910    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:05.936110 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.936065    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:28:05.938438 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:28:05.938412    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5b6052c_f79f_4707_8dda_60704d2bd6a6.slice/crio-95f5fcf476eadc7e022f884f4487a8d7f768fc0611639bdbdc1cd3e1de57a678 WatchSource:0}: Error finding container 95f5fcf476eadc7e022f884f4487a8d7f768fc0611639bdbdc1cd3e1de57a678: Status 404 returned error can't find the container with id 95f5fcf476eadc7e022f884f4487a8d7f768fc0611639bdbdc1cd3e1de57a678
Sep 16 14:28:05.940822 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.940791    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:05.940896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.940855    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:05.940896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:05.940883    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:06.802827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:06.802785    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerStarted","Data":"da76d8ed25815c9285cebe2c5d61b62c3a6a0ac6bc2b8ff61b20cfaa770607f7"}
Sep 16 14:28:06.803217 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:06.802888    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerStarted","Data":"95f5fcf476eadc7e022f884f4487a8d7f768fc0611639bdbdc1cd3e1de57a678"}
Sep 16 14:28:06.827453 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:06.827403    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podStartSLOduration=1.827388397 podStartE2EDuration="1.827388397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:05 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:06.82699143 +0000 UTC m=+1208.233571639" watchObservedRunningTime="2026-09-16 14:28:06.827388397 +0000 UTC m=+1208.233968605"
Sep 16 14:28:07.635354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:07.635265    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:07.640671 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:07.640645    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:07.809491 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:07.809461    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:07.810642 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:07.810620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:10.830175 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.830087    2580 generic.go:356] "Generic (PLEG): container finished" podID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerID="685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60" exitCode=0
Sep 16 14:28:10.830175 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.830132    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerDied","Data":"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"}
Sep 16 14:28:10.830619 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.830528    2580 scope.go:117] "RemoveContainer" containerID="685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"
Sep 16 14:28:10.831826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.831790    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:10.831897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.831882    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:10.831938 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:10.831921    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:11.838012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:11.837970    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerStarted","Data":"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"}
Sep 16 14:28:12.631906 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:12.631864    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:28:12.633016 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:12.632996    2580 patch_prober.go:28] interesting pod/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 16 14:28:12.633078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:12.633040    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 16 14:28:12.653340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:12.653311    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:28:13.631523 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:13.631488    2580 patch_prober.go:28] interesting pod/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 16 14:28:13.631976 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:13.631545    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 16 14:28:14.631140 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:14.631104    2580 patch_prober.go:28] interesting pod/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 16 14:28:14.631321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:14.631160    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 16 14:28:15.632024 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:15.631981    2580 patch_prober.go:28] interesting pod/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 16 14:28:15.632492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:15.632046    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 16 14:28:15.743764 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:15.743711    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:28:15.744019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:15.743988    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" podUID="5e223c0c-ac75-45ed-8761-ecc83024c147" containerName="payload-processing" containerID="cri-o://d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be" gracePeriod=30
Sep 16 14:28:16.111943 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.111917    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:28:16.139391 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.138175    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6dwlq\" (UniqueName: \"kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq\") pod \"5e223c0c-ac75-45ed-8761-ecc83024c147\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") "
Sep 16 14:28:16.139391 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.138330    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume\") pod \"5e223c0c-ac75-45ed-8761-ecc83024c147\" (UID: \"5e223c0c-ac75-45ed-8761-ecc83024c147\") "
Sep 16 14:28:16.139930 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.139882    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5e223c0c-ac75-45ed-8761-ecc83024c147" (UID: "5e223c0c-ac75-45ed-8761-ecc83024c147"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:16.150631 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.149808    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq" (OuterVolumeSpecName: "kube-api-access-6dwlq") pod "5e223c0c-ac75-45ed-8761-ecc83024c147" (UID: "5e223c0c-ac75-45ed-8761-ecc83024c147"). InnerVolumeSpecName "kube-api-access-6dwlq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:16.239676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.239645    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5e223c0c-ac75-45ed-8761-ecc83024c147-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:16.239676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.239673    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6dwlq\" (UniqueName: \"kubernetes.io/projected/5e223c0c-ac75-45ed-8761-ecc83024c147-kube-api-access-6dwlq\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:16.631798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.631711    2580 patch_prober.go:28] interesting pod/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body=
Sep 16 14:28:16.631798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.631779    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused"
Sep 16 14:28:16.865884 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.865851    2580 generic.go:356] "Generic (PLEG): container finished" podID="5e223c0c-ac75-45ed-8761-ecc83024c147" containerID="d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be" exitCode=0
Sep 16 14:28:16.866265 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.865903    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"
Sep 16 14:28:16.866265 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.865934    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" event={"ID":"5e223c0c-ac75-45ed-8761-ecc83024c147","Type":"ContainerDied","Data":"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"}
Sep 16 14:28:16.866265 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.865986    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2" event={"ID":"5e223c0c-ac75-45ed-8761-ecc83024c147","Type":"ContainerDied","Data":"a0ddba9d4fd90f1f2e97e6edc7428d4a1ffd744bdb1a981053319e476cb873c9"}
Sep 16 14:28:16.866265 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.866010    2580 scope.go:117] "RemoveContainer" containerID="d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"
Sep 16 14:28:16.884194 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.884092    2580 scope.go:117] "RemoveContainer" containerID="d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"
Sep 16 14:28:16.884695 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:16.884622    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be\": container with ID starting with d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be not found: ID does not exist" containerID="d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"
Sep 16 14:28:16.884695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.884660    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be"} err="failed to get container status \"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be\": rpc error: code = NotFound desc = could not find container \"d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be\": container with ID starting with d913e58290734a3b5cd51289278a31aad06c8d51f6250d0f2b5b0e6e458c00be not found: ID does not exist"
Sep 16 14:28:16.918829 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.918788    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:28:16.920919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:16.920878    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-5f53f6be-688549f475-f66c2"]
Sep 16 14:28:17.184900 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:17.184816    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5e223c0c-ac75-45ed-8761-ecc83024c147" path="/var/lib/kubelet/pods/5e223c0c-ac75-45ed-8761-ecc83024c147/volumes"
Sep 16 14:28:17.196918 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:17.196892    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:28:17.197149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:17.197123    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy" containerID="cri-o://d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746" gracePeriod=30
Sep 16 14:28:18.016833 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.016803    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:28:18.017226 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.017212    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5e223c0c-ac75-45ed-8761-ecc83024c147" containerName="payload-processing"
Sep 16 14:28:18.017226 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.017227    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5e223c0c-ac75-45ed-8761-ecc83024c147" containerName="payload-processing"
Sep 16 14:28:18.017319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.017310    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5e223c0c-ac75-45ed-8761-ecc83024c147" containerName="payload-processing"
Sep 16 14:28:18.020576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.020537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.024112 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.024080    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-377cc0d5-openshift-default-dockercfg-js94k\""
Sep 16 14:28:18.052904 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.052873    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:28:18.059344 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.059313    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.059547 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.059529    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.059686 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.059670    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lnzv5\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.059824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.059810    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.059977 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.059962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.060088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.060076    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.060456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.060435    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.060630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.060613    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.060736 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.060723    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.161849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161810    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.161849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161854    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161873    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161900    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161926    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.161956    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lnzv5\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162001    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162062    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162393 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162090    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162393 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162289    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162393 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162536 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162454    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162620 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162535    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.162860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.162842    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.164782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.164757    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.164927 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.164912    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.181733 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.181709    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.181916 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.181898    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lnzv5\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5\") pod \"e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") " pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.332154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.332050    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:18.610758 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.610679    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:28:18.613218 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:28:18.613189    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode51db172_c757_421b_94b1_3498f108a2f3.slice/crio-7bd9ca2e512800aeae70cbc54eeb777ebd60d2b1376e7fe32f48671f96d4b6cf WatchSource:0}: Error finding container 7bd9ca2e512800aeae70cbc54eeb777ebd60d2b1376e7fe32f48671f96d4b6cf: Status 404 returned error can't find the container with id 7bd9ca2e512800aeae70cbc54eeb777ebd60d2b1376e7fe32f48671f96d4b6cf
Sep 16 14:28:18.615847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.615815    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:18.615917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.615880    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:18.615917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.615910    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:18.881036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.880938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerStarted","Data":"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"}
Sep 16 14:28:18.881036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.881003    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerStarted","Data":"7bd9ca2e512800aeae70cbc54eeb777ebd60d2b1376e7fe32f48671f96d4b6cf"}
Sep 16 14:28:18.905517 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:18.905463    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podStartSLOduration=1.905444137 podStartE2EDuration="1.905444137s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:18.903383645 +0000 UTC m=+1220.309963888" watchObservedRunningTime="2026-09-16 14:28:18.905444137 +0000 UTC m=+1220.312024345"
Sep 16 14:28:19.332721 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:19.332678    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:19.334642 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:19.334612    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:19.334806 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:19.334706    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:20.337747 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:20.337715    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:20.893864 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:20.893823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:20.894770 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:20.894751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:22.442291 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.442259    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:28:22.509424 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509393    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509424 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509437    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509455    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509479    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509584    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509618    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509652    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.509753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509718    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2rt9k\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k\") pod \"a51fdc4e-035e-4442-aa96-dc14ec824829\" (UID: \"a51fdc4e-035e-4442-aa96-dc14ec824829\") "
Sep 16 14:28:22.510083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509772    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:22.510083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:22.510083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.509937    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:22.510241 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.510189    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.510241 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.510210    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.510241 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.510224    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.510401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.510243    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data" (OuterVolumeSpecName: "istio-data") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:22.510546 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.510522    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:22.512196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.512170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:28:22.512427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.512404    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token" (OuterVolumeSpecName: "istio-token") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:22.512595 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.512543    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k" (OuterVolumeSpecName: "kube-api-access-2rt9k") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "kube-api-access-2rt9k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:22.512595 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.512579    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "a51fdc4e-035e-4442-aa96-dc14ec824829" (UID: "a51fdc4e-035e-4442-aa96-dc14ec824829"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611300    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611334    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611343    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611351    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/a51fdc4e-035e-4442-aa96-dc14ec824829-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611359    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.611386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.611368    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2rt9k\" (UniqueName: \"kubernetes.io/projected/a51fdc4e-035e-4442-aa96-dc14ec824829-kube-api-access-2rt9k\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:22.907512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.907403    2580 generic.go:356] "Generic (PLEG): container finished" podID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerID="d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746" exitCode=0
Sep 16 14:28:22.907512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.907475    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"
Sep 16 14:28:22.907742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.907471    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerDied","Data":"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"}
Sep 16 14:28:22.907742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.907622    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x" event={"ID":"a51fdc4e-035e-4442-aa96-dc14ec824829","Type":"ContainerDied","Data":"b790bc786d9eb2eab1cbaf662173937d507dba312283ae307c6d859cee2f1f7e"}
Sep 16 14:28:22.907742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.907657    2580 scope.go:117] "RemoveContainer" containerID="d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"
Sep 16 14:28:22.917091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.917076    2580 scope.go:117] "RemoveContainer" containerID="685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"
Sep 16 14:28:22.925258 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.925240    2580 scope.go:117] "RemoveContainer" containerID="d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"
Sep 16 14:28:22.925524 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:22.925507    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746\": container with ID starting with d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746 not found: ID does not exist" containerID="d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"
Sep 16 14:28:22.925639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.925532    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746"} err="failed to get container status \"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746\": rpc error: code = NotFound desc = could not find container \"d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746\": container with ID starting with d61b20542442a369e763e07843cef49e44d1af47778b8a18a629632ff8152746 not found: ID does not exist"
Sep 16 14:28:22.925639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.925548    2580 scope.go:117] "RemoveContainer" containerID="685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"
Sep 16 14:28:22.925880 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:22.925860    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60\": container with ID starting with 685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60 not found: ID does not exist" containerID="685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"
Sep 16 14:28:22.925918 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.925885    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60"} err="failed to get container status \"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60\": rpc error: code = NotFound desc = could not find container \"685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60\": container with ID starting with 685dc1de05492e7cb0952f1d151931bb4a34bbdd5ed033c4969381f7dfd11e60 not found: ID does not exist"
Sep 16 14:28:22.934437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.934418    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:28:22.943190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:22.943169    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-5f53f6be-openshift-default-69bdf56c7b-b8f8x"]
Sep 16 14:28:23.184865 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.184774    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" path="/var/lib/kubelet/pods/a51fdc4e-035e-4442-aa96-dc14ec824829/volumes"
Sep 16 14:28:23.259604 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.259570    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:28:23.259981 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.259968    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.260033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.259982    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.260033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.260004    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.260033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.260009    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.260128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.260072    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.260128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.260082    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a51fdc4e-035e-4442-aa96-dc14ec824829" containerName="istio-proxy"
Sep 16 14:28:23.263041 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.263022    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.265242 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.265227    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-377cc0d5-dockercfg-rzlwx\""
Sep 16 14:28:23.265304 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.265281    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-377cc0d5\""
Sep 16 14:28:23.282429 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.282404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:28:23.284524 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.284501    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:28:23.287018 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.286997    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.302380 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.302354    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:28:23.317714 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.317686    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d9df\" (UniqueName: \"kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.317867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.317849    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.419286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.419247    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.419495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.419294    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.419495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.419364    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-knll8\" (UniqueName: \"kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.419495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.419391    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8d9df\" (UniqueName: \"kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.419961 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.419940    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.429757 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.429731    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d9df\" (UniqueName: \"kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df\") pod \"payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.520605 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.520541    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-knll8\" (UniqueName: \"kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.520983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.520692    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.521340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.521323    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.528930 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.528904    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-knll8\" (UniqueName: \"kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8\") pod \"payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") " pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.572959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.572933    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:23.599317 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.599282    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:23.857985 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.857960    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:28:23.860200 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:28:23.860172    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod871f9eee_a4f5_4a9f_8d7b_f2c15cfe181f.slice/crio-63880c235e055237ab6087fd97bf39877fa549a44586ffce03cbcd41ce2d4cd0 WatchSource:0}: Error finding container 63880c235e055237ab6087fd97bf39877fa549a44586ffce03cbcd41ce2d4cd0: Status 404 returned error can't find the container with id 63880c235e055237ab6087fd97bf39877fa549a44586ffce03cbcd41ce2d4cd0
Sep 16 14:28:23.886342 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.886321    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:28:23.896731 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:28:23.896704    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode43c2d80_5828_4947_bb42_8c12fadd47af.slice/crio-eac1266365e666f92ba854603341113e89bb8b56f7fb40ae27c9a838483576e7 WatchSource:0}: Error finding container eac1266365e666f92ba854603341113e89bb8b56f7fb40ae27c9a838483576e7: Status 404 returned error can't find the container with id eac1266365e666f92ba854603341113e89bb8b56f7fb40ae27c9a838483576e7
Sep 16 14:28:23.915377 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.915344    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" event={"ID":"e43c2d80-5828-4947-bb42-8c12fadd47af","Type":"ContainerStarted","Data":"eac1266365e666f92ba854603341113e89bb8b56f7fb40ae27c9a838483576e7"}
Sep 16 14:28:23.916545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:23.916523    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" event={"ID":"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f","Type":"ContainerStarted","Data":"63880c235e055237ab6087fd97bf39877fa549a44586ffce03cbcd41ce2d4cd0"}
Sep 16 14:28:24.923778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.923735    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" event={"ID":"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f","Type":"ContainerStarted","Data":"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"}
Sep 16 14:28:24.924204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.923827    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:24.925344 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.925319    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" event={"ID":"e43c2d80-5828-4947-bb42-8c12fadd47af","Type":"ContainerStarted","Data":"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"}
Sep 16 14:28:24.925472 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.925457    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:24.941999 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.941954    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" podStartSLOduration=1.941918381 podStartE2EDuration="1.941918381s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:24.940506847 +0000 UTC m=+1226.347087056" watchObservedRunningTime="2026-09-16 14:28:24.941918381 +0000 UTC m=+1226.348498592"
Sep 16 14:28:24.961716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:24.961670    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" podStartSLOduration=1.961656292 podStartE2EDuration="1.961656292s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:24.958203909 +0000 UTC m=+1226.364784118" watchObservedRunningTime="2026-09-16 14:28:24.961656292 +0000 UTC m=+1226.368236499"
Sep 16 14:28:29.846979 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:29.846947    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:28:29.855338 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:29.855309    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:28:35.933734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:35.933699    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:28:35.934112 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:35.933769    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:28:41.009579 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.008662    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:28:41.009579 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.008912    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" podUID="5041bc9f-861b-4457-910e-dcee3780e4f3" containerName="payload-processing" containerID="cri-o://e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955" gracePeriod=30
Sep 16 14:28:41.042621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.040096    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:28:41.042621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.040339    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" podUID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" containerName="payload-pre-processing" containerID="cri-o://7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272" gracePeriod=30
Sep 16 14:28:41.546784 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.546759    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:28:41.566698 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.566671    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:28:41.639960 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.639876    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wmqd5\" (UniqueName: \"kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5\") pod \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") "
Sep 16 14:28:41.640335 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.640314    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lcm6k\" (UniqueName: \"kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k\") pod \"5041bc9f-861b-4457-910e-dcee3780e4f3\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") "
Sep 16 14:28:41.640406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.640391    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume\") pod \"5041bc9f-861b-4457-910e-dcee3780e4f3\" (UID: \"5041bc9f-861b-4457-910e-dcee3780e4f3\") "
Sep 16 14:28:41.640599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.640578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume\") pod \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\" (UID: \"160a6fbc-2f10-4cd5-9140-b81c51b8da99\") "
Sep 16 14:28:41.640923 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.640895    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5041bc9f-861b-4457-910e-dcee3780e4f3" (UID: "5041bc9f-861b-4457-910e-dcee3780e4f3"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:41.640999 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.640953    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "160a6fbc-2f10-4cd5-9140-b81c51b8da99" (UID: "160a6fbc-2f10-4cd5-9140-b81c51b8da99"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:41.641069 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.641056    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/160a6fbc-2f10-4cd5-9140-b81c51b8da99-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:41.641107 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.641073    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5041bc9f-861b-4457-910e-dcee3780e4f3-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:41.642408 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.642379    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5" (OuterVolumeSpecName: "kube-api-access-wmqd5") pod "160a6fbc-2f10-4cd5-9140-b81c51b8da99" (UID: "160a6fbc-2f10-4cd5-9140-b81c51b8da99"). InnerVolumeSpecName "kube-api-access-wmqd5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:41.642484 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.642417    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k" (OuterVolumeSpecName: "kube-api-access-lcm6k") pod "5041bc9f-861b-4457-910e-dcee3780e4f3" (UID: "5041bc9f-861b-4457-910e-dcee3780e4f3"). InnerVolumeSpecName "kube-api-access-lcm6k". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:41.742427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.742376    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wmqd5\" (UniqueName: \"kubernetes.io/projected/160a6fbc-2f10-4cd5-9140-b81c51b8da99-kube-api-access-wmqd5\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:41.742427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:41.742417    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lcm6k\" (UniqueName: \"kubernetes.io/projected/5041bc9f-861b-4457-910e-dcee3780e4f3-kube-api-access-lcm6k\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:42.060748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.060513    2580 generic.go:356] "Generic (PLEG): container finished" podID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" containerID="7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272" exitCode=0
Sep 16 14:28:42.060748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.060582    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"
Sep 16 14:28:42.060748 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.060595    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" event={"ID":"160a6fbc-2f10-4cd5-9140-b81c51b8da99","Type":"ContainerDied","Data":"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"}
Sep 16 14:28:42.061197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.060885    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd" event={"ID":"160a6fbc-2f10-4cd5-9140-b81c51b8da99","Type":"ContainerDied","Data":"eca45b4ef20613babde350b1e2222e30efd3244c4047d72fffab9656ec42cdda"}
Sep 16 14:28:42.061197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.060973    2580 scope.go:117] "RemoveContainer" containerID="7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"
Sep 16 14:28:42.063017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.062861    2580 generic.go:356] "Generic (PLEG): container finished" podID="5041bc9f-861b-4457-910e-dcee3780e4f3" containerID="e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955" exitCode=0
Sep 16 14:28:42.063017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.062994    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" event={"ID":"5041bc9f-861b-4457-910e-dcee3780e4f3","Type":"ContainerDied","Data":"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"}
Sep 16 14:28:42.063164 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.063033    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj" event={"ID":"5041bc9f-861b-4457-910e-dcee3780e4f3","Type":"ContainerDied","Data":"579ec7a8a15c8b64bbe4c761f86230b274b6c9276add353dff7db06639cd6354"}
Sep 16 14:28:42.063164 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.062998    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"
Sep 16 14:28:42.086586 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.083842    2580 scope.go:117] "RemoveContainer" containerID="7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"
Sep 16 14:28:42.089583 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:42.088201    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272\": container with ID starting with 7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272 not found: ID does not exist" containerID="7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"
Sep 16 14:28:42.089583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.088240    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272"} err="failed to get container status \"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272\": rpc error: code = NotFound desc = could not find container \"7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272\": container with ID starting with 7bca24f4e463c8f2e3ca117ec68db553a6139ea0daa699782c400306c92d8272 not found: ID does not exist"
Sep 16 14:28:42.089583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.088265    2580 scope.go:117] "RemoveContainer" containerID="e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"
Sep 16 14:28:42.108945 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.108893    2580 scope.go:117] "RemoveContainer" containerID="e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"
Sep 16 14:28:42.109585 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:42.109540    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955\": container with ID starting with e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955 not found: ID does not exist" containerID="e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"
Sep 16 14:28:42.109769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.109743    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955"} err="failed to get container status \"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955\": rpc error: code = NotFound desc = could not find container \"e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955\": container with ID starting with e96984622910c28a857af027d54a0d491f10149015878e69d2323f63847db955 not found: ID does not exist"
Sep 16 14:28:42.119634 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.119611    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:28:42.122631 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.121977    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w3-d48266-7bc4dc4fc-lqqfj"]
Sep 16 14:28:42.130651 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.130622    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:28:42.132525 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:42.132506    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w3-d48266-578ffd5746-8qnmd"]
Sep 16 14:28:43.185075 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:43.184986    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" path="/var/lib/kubelet/pods/160a6fbc-2f10-4cd5-9140-b81c51b8da99/volumes"
Sep 16 14:28:43.185509 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:43.185408    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5041bc9f-861b-4457-910e-dcee3780e4f3" path="/var/lib/kubelet/pods/5041bc9f-861b-4457-910e-dcee3780e4f3/volumes"
Sep 16 14:28:44.866657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:44.866610    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerName="main" containerID="cri-o://150bbd7f2b42ab60cfdb7c4f6b6d2ba2df8c143418580ced1ea479f245fd089c" gracePeriod=45
Sep 16 14:28:44.880061 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:44.880021    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" podUID="30bd08aa-129f-4902-9596-3cf89ecec4b7" containerName="main" containerID="cri-o://60c9575752d5b926e5fac3e4109c7198241c6fe8b5b4274f765f30968726cadb" gracePeriod=45
Sep 16 14:28:45.012762 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.012691    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerName="main" probeResult="failure" output="Get \"https://10.133.0.39:8000/ready\": dial tcp 10.133.0.39:8000: connect: connection refused"
Sep 16 14:28:45.088669 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.088551    2580 generic.go:356] "Generic (PLEG): container finished" podID="30bd08aa-129f-4902-9596-3cf89ecec4b7" containerID="60c9575752d5b926e5fac3e4109c7198241c6fe8b5b4274f765f30968726cadb" exitCode=0
Sep 16 14:28:45.088860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.088683    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" event={"ID":"30bd08aa-129f-4902-9596-3cf89ecec4b7","Type":"ContainerDied","Data":"60c9575752d5b926e5fac3e4109c7198241c6fe8b5b4274f765f30968726cadb"}
Sep 16 14:28:45.090860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.090835    2580 generic.go:356] "Generic (PLEG): container finished" podID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerID="150bbd7f2b42ab60cfdb7c4f6b6d2ba2df8c143418580ced1ea479f245fd089c" exitCode=0
Sep 16 14:28:45.090978 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.090892    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" event={"ID":"0b68a47a-912e-43a1-a73a-809ee23417c0","Type":"ContainerDied","Data":"150bbd7f2b42ab60cfdb7c4f6b6d2ba2df8c143418580ced1ea479f245fd089c"}
Sep 16 14:28:45.146503 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.146473    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:28:45.178636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178603    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.178804 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178672    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.178804 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178753    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.178931 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178815    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zjkvs\" (UniqueName: \"kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.178931 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178847    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.178931 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.178910    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache\") pod \"0b68a47a-912e-43a1-a73a-809ee23417c0\" (UID: \"0b68a47a-912e-43a1-a73a-809ee23417c0\") "
Sep 16 14:28:45.180889 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.179454    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache" (OuterVolumeSpecName: "model-cache") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.180889 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.179632    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.180889 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.180271    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home" (OuterVolumeSpecName: "home") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.183314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.183283    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs" (OuterVolumeSpecName: "kube-api-access-zjkvs") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "kube-api-access-zjkvs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:45.183862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.183839    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm" (OuterVolumeSpecName: "dshm") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.184271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.184245    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "0b68a47a-912e-43a1-a73a-809ee23417c0" (UID: "0b68a47a-912e-43a1-a73a-809ee23417c0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:28:45.185017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.184997    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:28:45.280014 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.279981    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280029    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280112    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280139    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280203    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wsgn9\" (UniqueName: \"kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9\") pod \"30bd08aa-129f-4902-9596-3cf89ecec4b7\" (UID: \"30bd08aa-129f-4902-9596-3cf89ecec4b7\") "
Sep 16 14:28:45.280500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280324    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache" (OuterVolumeSpecName: "model-cache") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.280500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280361    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.280637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280527    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home" (OuterVolumeSpecName: "home") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.280689 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280675    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-home\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280696    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zjkvs\" (UniqueName: \"kubernetes.io/projected/0b68a47a-912e-43a1-a73a-809ee23417c0-kube-api-access-zjkvs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280712    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-home\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280725    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-model-cache\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280739    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-model-cache\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280752    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0b68a47a-912e-43a1-a73a-809ee23417c0-tls-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280766    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-tmp-dir\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280779    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-tmp-dir\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.280901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.280791    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/0b68a47a-912e-43a1-a73a-809ee23417c0-dshm\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.283054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.283018    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:28:45.283185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.283084    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm" (OuterVolumeSpecName: "dshm") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:45.283355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.283330    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9" (OuterVolumeSpecName: "kube-api-access-wsgn9") pod "30bd08aa-129f-4902-9596-3cf89ecec4b7" (UID: "30bd08aa-129f-4902-9596-3cf89ecec4b7"). InnerVolumeSpecName "kube-api-access-wsgn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:45.379169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379086    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"]
Sep 16 14:28:45.379787 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379767    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" containerName="payload-pre-processing"
Sep 16 14:28:45.379846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379793    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" containerName="payload-pre-processing"
Sep 16 14:28:45.379846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379806    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="30bd08aa-129f-4902-9596-3cf89ecec4b7" containerName="main"
Sep 16 14:28:45.379846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379816    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="30bd08aa-129f-4902-9596-3cf89ecec4b7" containerName="main"
Sep 16 14:28:45.379846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379830    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5041bc9f-861b-4457-910e-dcee3780e4f3" containerName="payload-processing"
Sep 16 14:28:45.379846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379840    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5041bc9f-861b-4457-910e-dcee3780e4f3" containerName="payload-processing"
Sep 16 14:28:45.380004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379854    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerName="main"
Sep 16 14:28:45.380004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379863    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerName="main"
Sep 16 14:28:45.380004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.379989    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5041bc9f-861b-4457-910e-dcee3780e4f3" containerName="payload-processing"
Sep 16 14:28:45.380095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.380004    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="160a6fbc-2f10-4cd5-9140-b81c51b8da99" containerName="payload-pre-processing"
Sep 16 14:28:45.380095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.380019    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="30bd08aa-129f-4902-9596-3cf89ecec4b7" containerName="main"
Sep 16 14:28:45.380095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.380031    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" containerName="main"
Sep 16 14:28:45.381927 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.381905    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/30bd08aa-129f-4902-9596-3cf89ecec4b7-tls-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.381989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.381931    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wsgn9\" (UniqueName: \"kubernetes.io/projected/30bd08aa-129f-4902-9596-3cf89ecec4b7-kube-api-access-wsgn9\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.381989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.381941    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/30bd08aa-129f-4902-9596-3cf89ecec4b7-dshm\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:45.383832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.383802    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:45.387437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.387227    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:28:45.387652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.387632    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:28:45.390286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.390261    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"]
Sep 16 14:28:45.483343 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.483290    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-377cc0d5-cf764\" (UID: \"7254f056-845a-40d3-a034-94c991abdf8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:45.584535 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.584485    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-377cc0d5-cf764\" (UID: \"7254f056-845a-40d3-a034-94c991abdf8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:45.585183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.585149    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-377cc0d5-cf764\" (UID: \"7254f056-845a-40d3-a034-94c991abdf8f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:45.699697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.699658    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:45.981311 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.981277    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"]
Sep 16 14:28:45.986495 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:28:45.986463    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7254f056_845a_40d3_a034_94c991abdf8f.slice/crio-98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5 WatchSource:0}: Error finding container 98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5: Status 404 returned error can't find the container with id 98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5
Sep 16 14:28:45.988828 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:45.988803    2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 16 14:28:46.098803 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.098762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764" event={"ID":"7254f056-845a-40d3-a034-94c991abdf8f","Type":"ContainerStarted","Data":"98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5"}
Sep 16 14:28:46.103171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.100286    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"
Sep 16 14:28:46.103171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.100292    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w" event={"ID":"30bd08aa-129f-4902-9596-3cf89ecec4b7","Type":"ContainerDied","Data":"bf7a008e99f079cde7278cd096506fa5711ef0085b37827d3acf87303f5c2ab9"}
Sep 16 14:28:46.103171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.100340    2580 scope.go:117] "RemoveContainer" containerID="60c9575752d5b926e5fac3e4109c7198241c6fe8b5b4274f765f30968726cadb"
Sep 16 14:28:46.103171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.101948    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"
Sep 16 14:28:46.103171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.101937    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75" event={"ID":"0b68a47a-912e-43a1-a73a-809ee23417c0","Type":"ContainerDied","Data":"3cef7c35f980752d548b99d0a1225735e74e98e081f9f7c7c42e4bb497a331cc"}
Sep 16 14:28:46.104089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.104066    2580 generic.go:356] "Generic (PLEG): container finished" podID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerID="da76d8ed25815c9285cebe2c5d61b62c3a6a0ac6bc2b8ff61b20cfaa770607f7" exitCode=0
Sep 16 14:28:46.104179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.104127    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerDied","Data":"da76d8ed25815c9285cebe2c5d61b62c3a6a0ac6bc2b8ff61b20cfaa770607f7"}
Sep 16 14:28:46.104934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.104896    2580 scope.go:117] "RemoveContainer" containerID="da76d8ed25815c9285cebe2c5d61b62c3a6a0ac6bc2b8ff61b20cfaa770607f7"
Sep 16 14:28:46.106228 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.106199    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:46.106326 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.106286    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:46.106395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.106339    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:46.111180 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.111162    2580 scope.go:117] "RemoveContainer" containerID="150bbd7f2b42ab60cfdb7c4f6b6d2ba2df8c143418580ced1ea479f245fd089c"
Sep 16 14:28:46.147790 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.147755    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:28:46.153277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.153251    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-2-w3-d48266-kserve-7745cbccf7-8xd75"]
Sep 16 14:28:46.164824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.164796    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:28:46.166907 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:46.166880    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-d48266/distinct-w3-d48266-kserve-566d849dbb-8l49w"]
Sep 16 14:28:47.113775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.113742    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerStarted","Data":"77fd84e6bbab43f150672101e528dfe115cbc2ac6fd49d9ac9da45d6cfa7f9e7"}
Sep 16 14:28:47.118395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.118260    2580 generic.go:356] "Generic (PLEG): container finished" podID="e51db172-c757-421b-94b1-3498f108a2f3" containerID="a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c" exitCode=0
Sep 16 14:28:47.118395 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.118307    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerDied","Data":"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"}
Sep 16 14:28:47.118772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.118754    2580 scope.go:117] "RemoveContainer" containerID="a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"
Sep 16 14:28:47.119984 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.119955    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:47.120093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.120039    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:47.120162 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.120089    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:28:47.190931 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.190530    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b68a47a-912e-43a1-a73a-809ee23417c0" path="/var/lib/kubelet/pods/0b68a47a-912e-43a1-a73a-809ee23417c0/volumes"
Sep 16 14:28:47.195633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.191167    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="30bd08aa-129f-4902-9596-3cf89ecec4b7" path="/var/lib/kubelet/pods/30bd08aa-129f-4902-9596-3cf89ecec4b7/volumes"
Sep 16 14:28:47.635849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.635766    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:47.636146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.636123    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 16 14:28:47.636216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:47.636173    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 16 14:28:48.128393 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.128355    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerStarted","Data":"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"}
Sep 16 14:28:48.332495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.332460    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:48.333878 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.333848    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:48.334008 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.333908    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:48.635782 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.635723    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 16 14:28:48.635979 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:48.635788    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 16 14:28:49.333237 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.333199    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:49.333764 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.333249    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:49.366322 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.366162    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:28:49.371774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.366413    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy" containerID="cri-o://4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801" gracePeriod=30
Sep 16 14:28:49.635743 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.635711    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 16 14:28:49.635983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:49.635961    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 16 14:28:50.140590 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.140539    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764" event={"ID":"7254f056-845a-40d3-a034-94c991abdf8f","Type":"ContainerStarted","Data":"d2fe66b1ff87059b975f803241c6a72e2edcbfd7d06d699c5e8dd1dd7dd8891a"}
Sep 16 14:28:50.158803 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.158757    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764" podStartSLOduration=1.755873555 podStartE2EDuration="5.158740765s" totalImagesPullingTime="3.40286721s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:28:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:28:50.156253314 +0000 UTC m=+1251.562833522" watchObservedRunningTime="2026-09-16 14:28:50.158740765 +0000 UTC m=+1251.565320976"
Sep 16 14:28:50.333201 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.333157    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:50.333375 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.333217    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:50.636305 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.636267    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 16 14:28:50.636474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.636329    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 16 14:28:50.894772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:50.894729    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:51.148011 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.147917    2580 generic.go:356] "Generic (PLEG): container finished" podID="7254f056-845a-40d3-a034-94c991abdf8f" containerID="d2fe66b1ff87059b975f803241c6a72e2edcbfd7d06d699c5e8dd1dd7dd8891a" exitCode=0
Sep 16 14:28:51.148011 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.147997    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764" event={"ID":"7254f056-845a-40d3-a034-94c991abdf8f","Type":"ContainerDied","Data":"d2fe66b1ff87059b975f803241c6a72e2edcbfd7d06d699c5e8dd1dd7dd8891a"}
Sep 16 14:28:51.333347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.333303    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:51.333516 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.333371    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:51.636056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.636017    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body=
Sep 16 14:28:51.636246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:51.636081    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused"
Sep 16 14:28:52.332716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.332684    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:52.332904 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.332738    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:52.358280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.358257    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:52.459829 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.459799    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca\") pod \"7254f056-845a-40d3-a034-94c991abdf8f\" (UID: \"7254f056-845a-40d3-a034-94c991abdf8f\") "
Sep 16 14:28:52.460153 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.460130    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "7254f056-845a-40d3-a034-94c991abdf8f" (UID: "7254f056-845a-40d3-a034-94c991abdf8f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:52.561312 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.561264    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/7254f056-845a-40d3-a034-94c991abdf8f-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:52.639958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.639929    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:52.640321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.640301    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:52.641034 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:52.641014    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:28:53.160113 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:53.160085    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"
Sep 16 14:28:53.160307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:53.160128    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764" event={"ID":"7254f056-845a-40d3-a034-94c991abdf8f","Type":"ContainerDied","Data":"98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5"}
Sep 16 14:28:53.160307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:53.160177    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="98b482d5d68511382ac35c949016cd2cd00a85af970b70e1f72d87963ab781c5"
Sep 16 14:28:53.332653 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:53.332613    2580 patch_prober.go:28] interesting pod/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body=
Sep 16 14:28:53.332861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:53.332674    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused"
Sep 16 14:28:54.336975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.336945    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:54.337891 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.337872    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:28:54.657363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.657339    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:28:54.693078 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693104    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693135    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693186    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693204    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693262    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693318    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2dx48\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693354    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693375    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo\") pod \"c245481b-2e49-41cf-92ae-b069df013a04\" (UID: \"c245481b-2e49-41cf-92ae-b069df013a04\") "
Sep 16 14:28:54.693477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693424    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:54.693742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693553    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:54.693742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693589    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data" (OuterVolumeSpecName: "istio-data") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:54.693742 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693477    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:28:54.693883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693783    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:54.693883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693849    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/c245481b-2e49-41cf-92ae-b069df013a04-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.693883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693869    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.693883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693881    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.694086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.693894    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.695663 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.695630    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:28:54.695788 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.695692    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token" (OuterVolumeSpecName: "istio-token") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:54.695917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.695899    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:28:54.696015 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.695996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48" (OuterVolumeSpecName: "kube-api-access-2dx48") pod "c245481b-2e49-41cf-92ae-b069df013a04" (UID: "c245481b-2e49-41cf-92ae-b069df013a04"). InnerVolumeSpecName "kube-api-access-2dx48". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:28:54.794998 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.794960    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.794998 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.794992    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/c245481b-2e49-41cf-92ae-b069df013a04-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.794998 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.795002    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.795240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.795011    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2dx48\" (UniqueName: \"kubernetes.io/projected/c245481b-2e49-41cf-92ae-b069df013a04-kube-api-access-2dx48\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:54.795240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:54.795019    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/c245481b-2e49-41cf-92ae-b069df013a04-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:28:55.172775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.172743    2580 generic.go:356] "Generic (PLEG): container finished" podID="c245481b-2e49-41cf-92ae-b069df013a04" containerID="4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801" exitCode=0
Sep 16 14:28:55.172955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.172790    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"
Sep 16 14:28:55.172955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.172830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerDied","Data":"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"}
Sep 16 14:28:55.172955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.172882    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5" event={"ID":"c245481b-2e49-41cf-92ae-b069df013a04","Type":"ContainerDied","Data":"b436d5552f3b2df84f6a8acf78d98217a8011843ba53392e9ff92d220c6a80a7"}
Sep 16 14:28:55.172955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.172906    2580 scope.go:117] "RemoveContainer" containerID="4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"
Sep 16 14:28:55.183221 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.183202    2580 scope.go:117] "RemoveContainer" containerID="2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"
Sep 16 14:28:55.191818 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.191802    2580 scope.go:117] "RemoveContainer" containerID="4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"
Sep 16 14:28:55.192078 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:55.192054    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801\": container with ID starting with 4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801 not found: ID does not exist" containerID="4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"
Sep 16 14:28:55.192174 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.192084    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801"} err="failed to get container status \"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801\": rpc error: code = NotFound desc = could not find container \"4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801\": container with ID starting with 4b31a219468f8979338bc9f5221acf10c65efa275f8cd2b92f24f06401cc5801 not found: ID does not exist"
Sep 16 14:28:55.192174 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.192101    2580 scope.go:117] "RemoveContainer" containerID="2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"
Sep 16 14:28:55.192346 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:28:55.192327    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c\": container with ID starting with 2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c not found: ID does not exist" containerID="2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"
Sep 16 14:28:55.192405 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.192355    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c"} err="failed to get container status \"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c\": rpc error: code = NotFound desc = could not find container \"2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c\": container with ID starting with 2e2e842fc8070aec566e21884750c5c47920197fc50e5889e66bfb5ca434279c not found: ID does not exist"
Sep 16 14:28:55.196052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.196030    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:28:55.205104 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:55.205080    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-d48266-openshift-default-668b554dfd-92zm5"]
Sep 16 14:28:57.185714 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:28:57.185681    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c245481b-2e49-41cf-92ae-b069df013a04" path="/var/lib/kubelet/pods/c245481b-2e49-41cf-92ae-b069df013a04/volumes"
Sep 16 14:29:02.252249 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.252202    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:29:02.252777 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.252448    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" podUID="e43c2d80-5828-4947-bb42-8c12fadd47af" containerName="payload-processing" containerID="cri-o://5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8" gracePeriod=30
Sep 16 14:29:02.267366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.267311    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:29:02.267971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.267907    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" podUID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" containerName="payload-pre-processing" containerID="cri-o://11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6" gracePeriod=30
Sep 16 14:29:02.545307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.545276    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:29:02.557148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.557112    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"]
Sep 16 14:29:02.561553 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.561522    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-377cc0d5-cf764"]
Sep 16 14:29:02.592207 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.592178    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:29:02.695129 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695094    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:29:02.695361 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d9df\" (UniqueName: \"kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df\") pod \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") "
Sep 16 14:29:02.695439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695373    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-knll8\" (UniqueName: \"kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8\") pod \"e43c2d80-5828-4947-bb42-8c12fadd47af\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") "
Sep 16 14:29:02.695439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695331    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy" containerID="cri-o://83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b" gracePeriod=30
Sep 16 14:29:02.695439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695425    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume\") pod \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\" (UID: \"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f\") "
Sep 16 14:29:02.695632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695456    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume\") pod \"e43c2d80-5828-4947-bb42-8c12fadd47af\" (UID: \"e43c2d80-5828-4947-bb42-8c12fadd47af\") "
Sep 16 14:29:02.695907 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.695881    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" (UID: "871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:29:02.696418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.696393    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e43c2d80-5828-4947-bb42-8c12fadd47af" (UID: "e43c2d80-5828-4947-bb42-8c12fadd47af"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:29:02.698119 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.698089    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8" (OuterVolumeSpecName: "kube-api-access-knll8") pod "e43c2d80-5828-4947-bb42-8c12fadd47af" (UID: "e43c2d80-5828-4947-bb42-8c12fadd47af"). InnerVolumeSpecName "kube-api-access-knll8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:02.699530 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.699507    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df" (OuterVolumeSpecName: "kube-api-access-8d9df") pod "871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" (UID: "871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f"). InnerVolumeSpecName "kube-api-access-8d9df". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:02.797415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.797161    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d9df\" (UniqueName: \"kubernetes.io/projected/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-kube-api-access-8d9df\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:02.797415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.797212    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-knll8\" (UniqueName: \"kubernetes.io/projected/e43c2d80-5828-4947-bb42-8c12fadd47af-kube-api-access-knll8\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:02.797415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.797226    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:02.797415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:02.797240    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e43c2d80-5828-4947-bb42-8c12fadd47af-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:03.184205 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.184126    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7254f056-845a-40d3-a034-94c991abdf8f" path="/var/lib/kubelet/pods/7254f056-845a-40d3-a034-94c991abdf8f/volumes"
Sep 16 14:29:03.219154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.219115    2580 generic.go:356] "Generic (PLEG): container finished" podID="e43c2d80-5828-4947-bb42-8c12fadd47af" containerID="5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8" exitCode=0
Sep 16 14:29:03.219341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.219161    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"
Sep 16 14:29:03.219341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.219204    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" event={"ID":"e43c2d80-5828-4947-bb42-8c12fadd47af","Type":"ContainerDied","Data":"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"}
Sep 16 14:29:03.219341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.219273    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h" event={"ID":"e43c2d80-5828-4947-bb42-8c12fadd47af","Type":"ContainerDied","Data":"eac1266365e666f92ba854603341113e89bb8b56f7fb40ae27c9a838483576e7"}
Sep 16 14:29:03.219341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.219305    2580 scope.go:117] "RemoveContainer" containerID="5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"
Sep 16 14:29:03.220666 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.220645    2580 generic.go:356] "Generic (PLEG): container finished" podID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" containerID="11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6" exitCode=0
Sep 16 14:29:03.220755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.220699    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"
Sep 16 14:29:03.220755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.220708    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" event={"ID":"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f","Type":"ContainerDied","Data":"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"}
Sep 16 14:29:03.220755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.220740    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl" event={"ID":"871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f","Type":"ContainerDied","Data":"63880c235e055237ab6087fd97bf39877fa549a44586ffce03cbcd41ce2d4cd0"}
Sep 16 14:29:03.228415 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.228397    2580 scope.go:117] "RemoveContainer" containerID="5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"
Sep 16 14:29:03.228724 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:29:03.228704    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8\": container with ID starting with 5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8 not found: ID does not exist" containerID="5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"
Sep 16 14:29:03.228814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.228736    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8"} err="failed to get container status \"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8\": rpc error: code = NotFound desc = could not find container \"5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8\": container with ID starting with 5d9fafa94d8d0c675e352e9d515fa714d1b7e982996321a77e878210bf4289f8 not found: ID does not exist"
Sep 16 14:29:03.228814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.228761    2580 scope.go:117] "RemoveContainer" containerID="11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"
Sep 16 14:29:03.236623 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.236604    2580 scope.go:117] "RemoveContainer" containerID="11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"
Sep 16 14:29:03.236922 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:29:03.236904    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6\": container with ID starting with 11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6 not found: ID does not exist" containerID="11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"
Sep 16 14:29:03.236986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.236929    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6"} err="failed to get container status \"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6\": rpc error: code = NotFound desc = could not find container \"11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6\": container with ID starting with 11bd928a93bbf59cb42c2723d8f355fcfade245accf4fc7dfe5be10d9b1b97a6 not found: ID does not exist"
Sep 16 14:29:03.241776 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.241753    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:29:03.245452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.245429    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-377cc0d5-7d8b644c46-hdktl"]
Sep 16 14:29:03.259039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.259010    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:29:03.268608 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:03.268580    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-377cc0d5-77b66d5665-h5z4h"]
Sep 16 14:29:05.184711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:05.184682    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" path="/var/lib/kubelet/pods/871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f/volumes"
Sep 16 14:29:05.185087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:05.185065    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e43c2d80-5828-4947-bb42-8c12fadd47af" path="/var/lib/kubelet/pods/e43c2d80-5828-4947-bb42-8c12fadd47af/volumes"
Sep 16 14:29:07.953835 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:07.953811    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:29:08.047781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047699    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.047781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.047781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047765    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.047781 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047787    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047873    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047894    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047920    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047935    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.047965    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lnzv5\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5\") pod \"e51db172-c757-421b-94b1-3498f108a2f3\" (UID: \"e51db172-c757-421b-94b1-3498f108a2f3\") "
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048016    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:29:08.048125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048091    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data" (OuterVolumeSpecName: "istio-data") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048127    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048325    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048404    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048424    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048438    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.048492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.048452    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:29:08.050027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.050006    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token" (OuterVolumeSpecName: "istio-token") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:08.050097 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.050038    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:29:08.050488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.050468    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:29:08.050607 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.050472    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5" (OuterVolumeSpecName: "kube-api-access-lnzv5") pod "e51db172-c757-421b-94b1-3498f108a2f3" (UID: "e51db172-c757-421b-94b1-3498f108a2f3"). InnerVolumeSpecName "kube-api-access-lnzv5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:29:08.149475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149428    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e51db172-c757-421b-94b1-3498f108a2f3-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.149475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149468    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.149475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149477    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lnzv5\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-kube-api-access-lnzv5\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.149475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149487    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e51db172-c757-421b-94b1-3498f108a2f3-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.149763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149495    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e51db172-c757-421b-94b1-3498f108a2f3-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.149763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.149503    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e51db172-c757-421b-94b1-3498f108a2f3-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:29:08.246682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.246647    2580 generic.go:356] "Generic (PLEG): container finished" podID="e51db172-c757-421b-94b1-3498f108a2f3" containerID="83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b" exitCode=0
Sep 16 14:29:08.246847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.246717    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"
Sep 16 14:29:08.246847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.246736    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerDied","Data":"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"}
Sep 16 14:29:08.246847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.246792    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm" event={"ID":"e51db172-c757-421b-94b1-3498f108a2f3","Type":"ContainerDied","Data":"7bd9ca2e512800aeae70cbc54eeb777ebd60d2b1376e7fe32f48671f96d4b6cf"}
Sep 16 14:29:08.246847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.246814    2580 scope.go:117] "RemoveContainer" containerID="83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"
Sep 16 14:29:08.255833 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.255814    2580 scope.go:117] "RemoveContainer" containerID="a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"
Sep 16 14:29:08.263836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.263819    2580 scope.go:117] "RemoveContainer" containerID="83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"
Sep 16 14:29:08.264099 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:29:08.264084    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b\": container with ID starting with 83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b not found: ID does not exist" containerID="83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"
Sep 16 14:29:08.264171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.264106    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b"} err="failed to get container status \"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b\": rpc error: code = NotFound desc = could not find container \"83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b\": container with ID starting with 83a4f98283b942beb5dfe428c6ffe42d63058dcad0e0a9821231100128a5007b not found: ID does not exist"
Sep 16 14:29:08.264171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.264121    2580 scope.go:117] "RemoveContainer" containerID="a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"
Sep 16 14:29:08.264349 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:29:08.264333    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c\": container with ID starting with a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c not found: ID does not exist" containerID="a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"
Sep 16 14:29:08.264386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.264358    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c"} err="failed to get container status \"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c\": rpc error: code = NotFound desc = could not find container \"a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c\": container with ID starting with a4f53907300fc673daf8c2d085ade56d48bd4064700f6889429ec30f5266d64c not found: ID does not exist"
Sep 16 14:29:08.275048 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.275013    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:29:08.276808 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:08.276791    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-377cc0d5-openshift-default-5479f84d8d-8tscm"]
Sep 16 14:29:09.185347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:09.185304    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e51db172-c757-421b-94b1-3498f108a2f3" path="/var/lib/kubelet/pods/e51db172-c757-421b-94b1-3498f108a2f3/volumes"
Sep 16 14:29:13.179262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:13.179218    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:29:40.592204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592169    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592531    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592543    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592572    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592580    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592592    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7254f056-845a-40d3-a034-94c991abdf8f" containerName="revoke-keys"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592597    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7254f056-845a-40d3-a034-94c991abdf8f" containerName="revoke-keys"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592604    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592609    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592615    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e43c2d80-5828-4947-bb42-8c12fadd47af" containerName="payload-processing"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592621    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e43c2d80-5828-4947-bb42-8c12fadd47af" containerName="payload-processing"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592630    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" containerName="payload-pre-processing"
Sep 16 14:29:40.592660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592635    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" containerName="payload-pre-processing"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592699    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7254f056-845a-40d3-a034-94c991abdf8f" containerName="revoke-keys"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592706    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592715    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592719    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592726    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="871f9eee-a4f5-4a9f-8d7b-f2c15cfe181f" containerName="payload-pre-processing"
Sep 16 14:29:40.593027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.592730    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="e43c2d80-5828-4947-bb42-8c12fadd47af" containerName="payload-processing"
Sep 16 14:29:40.595692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.595675    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.598832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.598805    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"default-dockercfg-nntwx\""
Sep 16 14:29:40.598974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.598805    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"auth-test-model-d31d7c-kserve-self-signed-certs\""
Sep 16 14:29:40.598974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.598814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"kube-root-ca.crt\""
Sep 16 14:29:40.598974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.598814    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w2-d31d7c\"/\"openshift-service-ca.crt\""
Sep 16 14:29:40.606583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.606542    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:29:40.640941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.640907    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.640941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.640942    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.641252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.640964    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5rrxx\" (UniqueName: \"kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.641252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.641064    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.641252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.641102    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.641252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.641232    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.741876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.741841    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.741901    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742032    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742074    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5rrxx\" (UniqueName: \"kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742260 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742100    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742260 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742129    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742395    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742473 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742438    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.742529 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.742510    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.744222 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.744200    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.744383 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.744365    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.750083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.750064    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5rrxx\" (UniqueName: \"kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx\") pod \"auth-test-model-d31d7c-kserve-6d6d495995-mjq79\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") " pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:40.906540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:40.906441    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:41.177629 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:41.177603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:29:41.179864 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:29:41.179829    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod3ba1bb49_4fd4_4893_b355_740e404748d6.slice/crio-c792198b20567a9dada5d5b21b7e931ba2da4ba265ce3db94a57845ec083673a WatchSource:0}: Error finding container c792198b20567a9dada5d5b21b7e931ba2da4ba265ce3db94a57845ec083673a: Status 404 returned error can't find the container with id c792198b20567a9dada5d5b21b7e931ba2da4ba265ce3db94a57845ec083673a
Sep 16 14:29:41.406275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:41.406237    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" event={"ID":"3ba1bb49-4fd4-4893-b355-740e404748d6","Type":"ContainerStarted","Data":"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"}
Sep 16 14:29:41.406452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:41.406285    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" event={"ID":"3ba1bb49-4fd4-4893-b355-740e404748d6","Type":"ContainerStarted","Data":"c792198b20567a9dada5d5b21b7e931ba2da4ba265ce3db94a57845ec083673a"}
Sep 16 14:29:41.422681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:41.422600    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" podStartSLOduration=1.422582556 podStartE2EDuration="1.422582556s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:29:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:29:41.422455718 +0000 UTC m=+1302.829035927" watchObservedRunningTime="2026-09-16 14:29:41.422582556 +0000 UTC m=+1302.829162764"
Sep 16 14:29:50.907609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:50.907536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:50.908036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:50.907621    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:50.920638 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:50.920604    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:29:51.468667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:29:51.468637    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:30:00.179171 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.179135    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"]
Sep 16 14:30:00.179723 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.179703    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:30:00.179816 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.179726    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="e51db172-c757-421b-94b1-3498f108a2f3" containerName="istio-proxy"
Sep 16 14:30:00.179868 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.179841    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c245481b-2e49-41cf-92ae-b069df013a04" containerName="istio-proxy"
Sep 16 14:30:00.184226 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.184205    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"
Sep 16 14:30:00.186911 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.186889    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:30:00.193733 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.193718    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"
Sep 16 14:30:00.203897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.203856    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"]
Sep 16 14:30:00.208115 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.208095    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"]
Sep 16 14:30:00.208229 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.208203    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"
Sep 16 14:30:00.228788 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.228745    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"]
Sep 16 14:30:00.237531 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.237510    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"
Sep 16 14:30:00.477653 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.477624    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"]
Sep 16 14:30:00.480333 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:30:00.480307    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d03659d_3de2_4cd1_b426_ac79ae672f33.slice/crio-f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543 WatchSource:0}: Error finding container f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543: Status 404 returned error can't find the container with id f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543
Sep 16 14:30:00.504615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.504589    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp" event={"ID":"7d03659d-3de2-4cd1-b426-ac79ae672f33","Type":"ContainerStarted","Data":"f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543"}
Sep 16 14:30:00.521304 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.521278    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"]
Sep 16 14:30:00.523113 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:30:00.523090    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae1eab06_146f_4f0a_bb28_bb2e6971ada2.slice/crio-9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f WatchSource:0}: Error finding container 9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f: Status 404 returned error can't find the container with id 9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f
Sep 16 14:30:00.788469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.788445    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:30:00.792088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.792070    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.794289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.794271    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"kube-root-ca.crt\""
Sep 16 14:30:00.794943 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.794921    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"openshift-service-ca.crt\""
Sep 16 14:30:00.795056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.794999    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"default-dockercfg-rnmfp\""
Sep 16 14:30:00.795056 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.795006    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-11636c\"/\"embedding-w0-11636c-kserve-self-signed-certs\""
Sep 16 14:30:00.809147 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.809125    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:30:00.838390 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838357    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.838545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838437    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.838545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.838545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838519    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w97vt\" (UniqueName: \"kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.838733 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.838733 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.838615    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939507    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939544    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939791 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939611    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939791 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939689    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939791 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939753    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.939791 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939779    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w97vt\" (UniqueName: \"kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.940002 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.939974    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.940061 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.940044    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.940168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.940132    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.943156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.943093    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.943624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.943605    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:00.949702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:00.949671    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-w97vt\" (UniqueName: \"kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt\") pod \"embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") " pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:01.101842 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.101755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:01.374940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.374885    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:30:01.377277 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:30:01.377246    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod18e1a6b5_fe93_4274_900e_579edec5830c.slice/crio-63f85833181cbd7eebdb7940025befb5fe352d6f015e8ad0733db38e5fde24ef WatchSource:0}: Error finding container 63f85833181cbd7eebdb7940025befb5fe352d6f015e8ad0733db38e5fde24ef: Status 404 returned error can't find the container with id 63f85833181cbd7eebdb7940025befb5fe352d6f015e8ad0733db38e5fde24ef
Sep 16 14:30:01.513372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.513320    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" event={"ID":"18e1a6b5-fe93-4274-900e-579edec5830c","Type":"ContainerStarted","Data":"812d2cedd4e113b638aa5f89809529806aee3a0c966dfd105fd819c92b40e1f2"}
Sep 16 14:30:01.513372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.513374    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" event={"ID":"18e1a6b5-fe93-4274-900e-579edec5830c","Type":"ContainerStarted","Data":"63f85833181cbd7eebdb7940025befb5fe352d6f015e8ad0733db38e5fde24ef"}
Sep 16 14:30:01.515055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.515026    2580 generic.go:356] "Generic (PLEG): container finished" podID="ae1eab06-146f-4f0a-bb28-bb2e6971ada2" containerID="1fff9fba5661a4d5f1eba92e2a6f51483a5d6c4d4eb2af31d285db11a50bf8ed" exitCode=0
Sep 16 14:30:01.515182 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.515107    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx" event={"ID":"ae1eab06-146f-4f0a-bb28-bb2e6971ada2","Type":"ContainerDied","Data":"1fff9fba5661a4d5f1eba92e2a6f51483a5d6c4d4eb2af31d285db11a50bf8ed"}
Sep 16 14:30:01.515182 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.515162    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx" event={"ID":"ae1eab06-146f-4f0a-bb28-bb2e6971ada2","Type":"ContainerStarted","Data":"9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f"}
Sep 16 14:30:01.516632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.516608    2580 generic.go:356] "Generic (PLEG): container finished" podID="7d03659d-3de2-4cd1-b426-ac79ae672f33" containerID="09998e15862f9088733cb18db2b3bdd67326a31207977c3227082d1d259f364d" exitCode=0
Sep 16 14:30:01.516756 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.516730    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp" event={"ID":"7d03659d-3de2-4cd1-b426-ac79ae672f33","Type":"ContainerDied","Data":"09998e15862f9088733cb18db2b3bdd67326a31207977c3227082d1d259f364d"}
Sep 16 14:30:01.532796 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:01.532743    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" podStartSLOduration=1.5327223079999999 podStartE2EDuration="1.532722308s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:01.530316645 +0000 UTC m=+1322.936896868" watchObservedRunningTime="2026-09-16 14:30:01.532722308 +0000 UTC m=+1322.939302518"
Sep 16 14:30:02.658974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:02.658946    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"
Sep 16 14:30:02.689697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:02.689672    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"
Sep 16 14:30:03.527418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.527337    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"
Sep 16 14:30:03.527418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.527359    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx" event={"ID":"ae1eab06-146f-4f0a-bb28-bb2e6971ada2","Type":"ContainerDied","Data":"9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f"}
Sep 16 14:30:03.527418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.527404    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9d422090ad807f09a2939b1651d784b0c325f74cf11fe8465c435e577cf0162f"
Sep 16 14:30:03.528696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.528678    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"
Sep 16 14:30:03.528836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.528706    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp" event={"ID":"7d03659d-3de2-4cd1-b426-ac79ae672f33","Type":"ContainerDied","Data":"f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543"}
Sep 16 14:30:03.528836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:03.528743    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f863987a5a50336c9e1c13a2be9999faad42e827676768d6b7421cfcf58b8543"
Sep 16 14:30:11.102409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:11.102369    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:11.102871 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:11.102453    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:11.115204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:11.115180    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:11.582449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:11.582418    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:30:41.017623 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.017582    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.017987    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d03659d-3de2-4cd1-b426-ac79ae672f33" containerName="cleanup"
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.018000    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d03659d-3de2-4cd1-b426-ac79ae672f33" containerName="cleanup"
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.018007    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ae1eab06-146f-4f0a-bb28-bb2e6971ada2" containerName="cleanup"
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.018013    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ae1eab06-146f-4f0a-bb28-bb2e6971ada2" containerName="cleanup"
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.018080    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ae1eab06-146f-4f0a-bb28-bb2e6971ada2" containerName="cleanup"
Sep 16 14:30:41.018148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.018091    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d03659d-3de2-4cd1-b426-ac79ae672f33" containerName="cleanup"
Sep 16 14:30:41.021278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.021260    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.023949 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.023926    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-2524a893-openshift-default-dockercfg-zstbd\""
Sep 16 14:30:41.034632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.034603    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:30:41.129913 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.129875    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.129937    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.129993    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5wfb\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130066    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130097    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130180    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130274    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130302    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.130382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.130335    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231739 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231701    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231758    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231795    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231814    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231837    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231876    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231916    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.231948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231937    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q5wfb\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.231990    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.232157    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.232258    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.232366    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232477 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.232425    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.232695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.232676    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.234187 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.234165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.234462 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.234444    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.241048 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.241028    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.244986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.244963    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5wfb\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb\") pod \"e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") " pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.333654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.333542    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:41.611001 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.610970    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:30:41.613171 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:30:41.613146    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c35a036_770d_40a3_b07e_8dac463f1052.slice/crio-aa2b6056170f0b3e958da6333b00f700e5fbf0a5a401949071feebed32c73736 WatchSource:0}: Error finding container aa2b6056170f0b3e958da6333b00f700e5fbf0a5a401949071feebed32c73736: Status 404 returned error can't find the container with id aa2b6056170f0b3e958da6333b00f700e5fbf0a5a401949071feebed32c73736
Sep 16 14:30:41.615642 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.615611    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:30:41.615709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.615671    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:30:41.615709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.615697    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:30:41.728406 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.728368    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerStarted","Data":"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"}
Sep 16 14:30:41.728615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.728424    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerStarted","Data":"aa2b6056170f0b3e958da6333b00f700e5fbf0a5a401949071feebed32c73736"}
Sep 16 14:30:41.762592 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:41.759998    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podStartSLOduration=1.759972867 podStartE2EDuration="1.759972867s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:41.757032015 +0000 UTC m=+1363.163612218" watchObservedRunningTime="2026-09-16 14:30:41.759972867 +0000 UTC m=+1363.166553078"
Sep 16 14:30:42.334509 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:42.334466    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:42.340096 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:42.340069    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:42.733702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:42.733669    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:42.734768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:42.734751    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:30:43.178467 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:43.178378    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:30:47.025797 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.025761    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:30:47.029177 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.029160    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.031572 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.031540    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-2524a893-dockercfg-srj2x\""
Sep 16 14:30:47.032328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.032309    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-2524a893\""
Sep 16 14:30:47.059412 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.059380    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:30:47.074708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.074675    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:30:47.078149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.078129    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.090530 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.090508    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.090657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.090540    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zbpt7\" (UniqueName: \"kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.095976 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.095952    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:30:47.191830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.191794    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.191830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.191839    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zbpt7\" (UniqueName: \"kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.192089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.191909    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.192089 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.191926    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whrt9\" (UniqueName: \"kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.192474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.192455    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.202661 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.202636    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zbpt7\" (UniqueName: \"kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7\") pod \"payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.292625 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.292511    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.292625 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.292547    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whrt9\" (UniqueName: \"kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.293268 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.293246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.301657 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.301635    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whrt9\" (UniqueName: \"kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9\") pod \"payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") " pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.340325 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.340287    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.388250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.388216    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.617948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.617915    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:30:47.677182 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.677153    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:30:47.680710 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:30:47.680682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod894bad99_4db0_45ec_9c68_bc4be62f9d71.slice/crio-b628954342dc9a3ca11d957563f7651136111407d0ba83a5dee0a0b374a14c3b WatchSource:0}: Error finding container b628954342dc9a3ca11d957563f7651136111407d0ba83a5dee0a0b374a14c3b: Status 404 returned error can't find the container with id b628954342dc9a3ca11d957563f7651136111407d0ba83a5dee0a0b374a14c3b
Sep 16 14:30:47.760495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.760460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" event={"ID":"497286c6-d963-4d39-81ae-f5fafa8e0ad2","Type":"ContainerStarted","Data":"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"}
Sep 16 14:30:47.760696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.760519    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" event={"ID":"497286c6-d963-4d39-81ae-f5fafa8e0ad2","Type":"ContainerStarted","Data":"bc3ae0f8e9b22759a00fff84b1735407310c62f5c0f49ef783cab57bcc440947"}
Sep 16 14:30:47.760696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.760627    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:47.762443 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.762417    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" event={"ID":"894bad99-4db0-45ec-9c68-bc4be62f9d71","Type":"ContainerStarted","Data":"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"}
Sep 16 14:30:47.762584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.762460    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" event={"ID":"894bad99-4db0-45ec-9c68-bc4be62f9d71","Type":"ContainerStarted","Data":"b628954342dc9a3ca11d957563f7651136111407d0ba83a5dee0a0b374a14c3b"}
Sep 16 14:30:47.762584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.762509    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:30:47.780486 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.780053    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" podStartSLOduration=1.78002934 podStartE2EDuration="1.78002934s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:47.778314796 +0000 UTC m=+1369.184895005" watchObservedRunningTime="2026-09-16 14:30:47.78002934 +0000 UTC m=+1369.186609550"
Sep 16 14:30:47.809449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:47.809388    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" podStartSLOduration=0.809370028 podStartE2EDuration="809.370028ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:30:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:30:47.807188246 +0000 UTC m=+1369.213768473" watchObservedRunningTime="2026-09-16 14:30:47.809370028 +0000 UTC m=+1369.215950239"
Sep 16 14:30:58.771377 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:58.771333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:30:58.772763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:30:58.772735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:31:03.932582 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:03.931882    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"]
Sep 16 14:31:03.937585 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:03.937543    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:03.944905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:03.944873    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:31:03.945022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:03.944873    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:31:03.956859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:03.956833    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"]
Sep 16 14:31:04.065433 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.065391    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6\" (UID: \"be27ef0f-6e14-4b6d-a57e-d9013299f9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:04.166940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.166904    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6\" (UID: \"be27ef0f-6e14-4b6d-a57e-d9013299f9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:04.167495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.167474    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6\" (UID: \"be27ef0f-6e14-4b6d-a57e-d9013299f9b0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:04.247899 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.247870    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:04.525917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.525891    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"]
Sep 16 14:31:04.527745 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:31:04.527713    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe27ef0f_6e14_4b6d_a57e_d9013299f9b0.slice/crio-96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173 WatchSource:0}: Error finding container 96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173: Status 404 returned error can't find the container with id 96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173
Sep 16 14:31:04.856219 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:04.856181    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6" event={"ID":"be27ef0f-6e14-4b6d-a57e-d9013299f9b0","Type":"ContainerStarted","Data":"96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173"}
Sep 16 14:31:05.863716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.863683    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c35a036-770d-40a3-b07e-8dac463f1052" containerID="eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6" exitCode=0
Sep 16 14:31:05.864219 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.863762    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerDied","Data":"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"}
Sep 16 14:31:05.864291 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.864244    2580 scope.go:117] "RemoveContainer" containerID="eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"
Sep 16 14:31:05.865534 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.865501    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:31:05.865652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.865577    2580 generic.go:356] "Generic (PLEG): container finished" podID="be27ef0f-6e14-4b6d-a57e-d9013299f9b0" containerID="5828acdedcf6fb96477ba7ea461407bf7dd49b6bea144dbc7e5d1a53cb12ceba" exitCode=0
Sep 16 14:31:05.865652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.865610    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:31:05.865738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.865658    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:31:05.865775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:05.865658    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6" event={"ID":"be27ef0f-6e14-4b6d-a57e-d9013299f9b0","Type":"ContainerDied","Data":"5828acdedcf6fb96477ba7ea461407bf7dd49b6bea144dbc7e5d1a53cb12ceba"}
Sep 16 14:31:06.882697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:06.882656    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerStarted","Data":"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"}
Sep 16 14:31:07.046876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.046852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:07.199365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.199252    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca\") pod \"be27ef0f-6e14-4b6d-a57e-d9013299f9b0\" (UID: \"be27ef0f-6e14-4b6d-a57e-d9013299f9b0\") "
Sep 16 14:31:07.199746 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.199623    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "be27ef0f-6e14-4b6d-a57e-d9013299f9b0" (UID: "be27ef0f-6e14-4b6d-a57e-d9013299f9b0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:07.200167 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.200078    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/be27ef0f-6e14-4b6d-a57e-d9013299f9b0-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:07.334289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.334254    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:07.334503 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.334485    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 14:31:07.334581 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.334537    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 14:31:07.893146 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.893113    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"
Sep 16 14:31:07.893674 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.893167    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6" event={"ID":"be27ef0f-6e14-4b6d-a57e-d9013299f9b0","Type":"ContainerDied","Data":"96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173"}
Sep 16 14:31:07.893674 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:07.893233    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="96236cabb6a69e2dc9931dda6c5291356a0bcf1f97a25159fcb69071bb704173"
Sep 16 14:31:08.333811 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:08.333778    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 14:31:08.334000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:08.333828    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 14:31:08.800138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:08.800101    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:31:09.334500 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:09.334463    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 14:31:09.334914 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:09.334524    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 14:31:10.334380 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:10.334342    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 14:31:10.334602 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:10.334411    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 14:31:11.334401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:11.334366    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused" start-of-body=
Sep 16 14:31:11.334636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:11.334419    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.55:15021/healthz/ready\": dial tcp 10.133.0.55:15021: connect: connection refused"
Sep 16 14:31:12.338397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:12.338361    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:12.338869 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:12.338670    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:12.339337 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:12.339318    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:16.654080 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.654043    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"]
Sep 16 14:31:16.654456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.654438    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="be27ef0f-6e14-4b6d-a57e-d9013299f9b0" containerName="revoke-keys"
Sep 16 14:31:16.654456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.654449    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="be27ef0f-6e14-4b6d-a57e-d9013299f9b0" containerName="revoke-keys"
Sep 16 14:31:16.654535 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.654529    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="be27ef0f-6e14-4b6d-a57e-d9013299f9b0" containerName="revoke-keys"
Sep 16 14:31:16.656772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.656755    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:16.660176 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.660154    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:31:16.660481 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.660461    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:31:16.672425 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.672396    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"]
Sep 16 14:31:16.797858 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.797815    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2524a893-tbchs\" (UID: \"55854f11-049a-44fa-a617-1391555207f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:16.899060 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.899020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2524a893-tbchs\" (UID: \"55854f11-049a-44fa-a617-1391555207f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:16.899659 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.899638    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-2524a893-tbchs\" (UID: \"55854f11-049a-44fa-a617-1391555207f1\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:16.966780 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:16.966734    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:17.255375 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:17.255350    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"]
Sep 16 14:31:17.258217 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:31:17.258185    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55854f11_049a_44fa_a617_1391555207f1.slice/crio-797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281 WatchSource:0}: Error finding container 797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281: Status 404 returned error can't find the container with id 797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281
Sep 16 14:31:17.948681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:17.948638    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs" event={"ID":"55854f11-049a-44fa-a617-1391555207f1","Type":"ContainerStarted","Data":"e3d36d584b1606b67d6c982fd6a9a442f294ab853912eb826da2a2e631c796df"}
Sep 16 14:31:17.948681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:17.948693    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs" event={"ID":"55854f11-049a-44fa-a617-1391555207f1","Type":"ContainerStarted","Data":"797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281"}
Sep 16 14:31:17.976208 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:17.976162    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs" podStartSLOduration=1.976142493 podStartE2EDuration="1.976142493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:31:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:31:17.974786433 +0000 UTC m=+1399.381366640" watchObservedRunningTime="2026-09-16 14:31:17.976142493 +0000 UTC m=+1399.382722701"
Sep 16 14:31:18.280676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.280551    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:31:18.281287 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.281214    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" podUID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" containerName="payload-processing" containerID="cri-o://2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7" gracePeriod=30
Sep 16 14:31:18.290186 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.290154    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:31:18.290435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.290386    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" podUID="3920cc91-0b37-4141-bf1f-fab93c9977b9" containerName="payload-pre-processing" containerID="cri-o://291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06" gracePeriod=30
Sep 16 14:31:18.583696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.583669    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:31:18.586660 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.586639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:31:18.688354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.688317    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"]
Sep 16 14:31:18.698770 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.696681    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-11636c-2mkf6"]
Sep 16 14:31:18.719051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719018    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sn8pf\" (UniqueName: \"kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf\") pod \"3920cc91-0b37-4141-bf1f-fab93c9977b9\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") "
Sep 16 14:31:18.719051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719052    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pg8fr\" (UniqueName: \"kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr\") pod \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") "
Sep 16 14:31:18.719275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719118    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume\") pod \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\" (UID: \"50cbf182-f238-4b7b-8cd1-4baf4c56952b\") "
Sep 16 14:31:18.719275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume\") pod \"3920cc91-0b37-4141-bf1f-fab93c9977b9\" (UID: \"3920cc91-0b37-4141-bf1f-fab93c9977b9\") "
Sep 16 14:31:18.719643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719613    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "50cbf182-f238-4b7b-8cd1-4baf4c56952b" (UID: "50cbf182-f238-4b7b-8cd1-4baf4c56952b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:18.719643 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.719627    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3920cc91-0b37-4141-bf1f-fab93c9977b9" (UID: "3920cc91-0b37-4141-bf1f-fab93c9977b9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:18.721283 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.721259    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr" (OuterVolumeSpecName: "kube-api-access-pg8fr") pod "50cbf182-f238-4b7b-8cd1-4baf4c56952b" (UID: "50cbf182-f238-4b7b-8cd1-4baf4c56952b"). InnerVolumeSpecName "kube-api-access-pg8fr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:18.721377 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.721287    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf" (OuterVolumeSpecName: "kube-api-access-sn8pf") pod "3920cc91-0b37-4141-bf1f-fab93c9977b9" (UID: "3920cc91-0b37-4141-bf1f-fab93c9977b9"). InnerVolumeSpecName "kube-api-access-sn8pf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:18.821151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.821070    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3920cc91-0b37-4141-bf1f-fab93c9977b9-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:18.821151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.821100    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sn8pf\" (UniqueName: \"kubernetes.io/projected/3920cc91-0b37-4141-bf1f-fab93c9977b9-kube-api-access-sn8pf\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:18.821151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.821110    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pg8fr\" (UniqueName: \"kubernetes.io/projected/50cbf182-f238-4b7b-8cd1-4baf4c56952b-kube-api-access-pg8fr\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:18.821151 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.821119    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/50cbf182-f238-4b7b-8cd1-4baf4c56952b-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:18.956363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.956329    2580 generic.go:356] "Generic (PLEG): container finished" podID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" containerID="2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7" exitCode=0
Sep 16 14:31:18.956850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.956385    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"
Sep 16 14:31:18.956850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.956407    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" event={"ID":"50cbf182-f238-4b7b-8cd1-4baf4c56952b","Type":"ContainerDied","Data":"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"}
Sep 16 14:31:18.956850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.956482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7" event={"ID":"50cbf182-f238-4b7b-8cd1-4baf4c56952b","Type":"ContainerDied","Data":"994dcef474f264ec3f0d3aaf598685367fe1d0e4057a7040a5797352873b9689"}
Sep 16 14:31:18.956850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.956514    2580 scope.go:117] "RemoveContainer" containerID="2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"
Sep 16 14:31:18.958115 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.958060    2580 generic.go:356] "Generic (PLEG): container finished" podID="55854f11-049a-44fa-a617-1391555207f1" containerID="e3d36d584b1606b67d6c982fd6a9a442f294ab853912eb826da2a2e631c796df" exitCode=0
Sep 16 14:31:18.958227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.958138    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs" event={"ID":"55854f11-049a-44fa-a617-1391555207f1","Type":"ContainerDied","Data":"e3d36d584b1606b67d6c982fd6a9a442f294ab853912eb826da2a2e631c796df"}
Sep 16 14:31:18.959571 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.959538    2580 generic.go:356] "Generic (PLEG): container finished" podID="3920cc91-0b37-4141-bf1f-fab93c9977b9" containerID="291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06" exitCode=0
Sep 16 14:31:18.959676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.959572    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" event={"ID":"3920cc91-0b37-4141-bf1f-fab93c9977b9","Type":"ContainerDied","Data":"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"}
Sep 16 14:31:18.959727 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.959700    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx" event={"ID":"3920cc91-0b37-4141-bf1f-fab93c9977b9","Type":"ContainerDied","Data":"d4276cc98ba1ea9753e2200d9efce1597367c2c18a6a4ae7c72edfc7cc6901ea"}
Sep 16 14:31:18.959727 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.959600    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"
Sep 16 14:31:18.967269 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.967249    2580 scope.go:117] "RemoveContainer" containerID="2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"
Sep 16 14:31:18.967606 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:18.967583    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7\": container with ID starting with 2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7 not found: ID does not exist" containerID="2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"
Sep 16 14:31:18.967670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.967615    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7"} err="failed to get container status \"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7\": rpc error: code = NotFound desc = could not find container \"2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7\": container with ID starting with 2987d547f2b73be354fd551ac6aee61e814d472e00c2ed1fb9d2b2b3eb18f5e7 not found: ID does not exist"
Sep 16 14:31:18.967670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.967633    2580 scope.go:117] "RemoveContainer" containerID="291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"
Sep 16 14:31:18.975776 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.975755    2580 scope.go:117] "RemoveContainer" containerID="291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"
Sep 16 14:31:18.976032 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:18.976017    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06\": container with ID starting with 291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06 not found: ID does not exist" containerID="291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"
Sep 16 14:31:18.976083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:18.976038    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06"} err="failed to get container status \"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06\": rpc error: code = NotFound desc = could not find container \"291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06\": container with ID starting with 291511201eb107d17e455b14e9222ae05b6dd0542e4fdc737c418f2aead2ab06 not found: ID does not exist"
Sep 16 14:31:19.003204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.003168    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:31:19.007122 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.007090    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-11636c-7668479d58-h7hcx"]
Sep 16 14:31:19.019467 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.019436    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:31:19.027991 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.027964    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-worker-w0-11636c-6896b9f79-gglc7"]
Sep 16 14:31:19.185254 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.185169    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3920cc91-0b37-4141-bf1f-fab93c9977b9" path="/var/lib/kubelet/pods/3920cc91-0b37-4141-bf1f-fab93c9977b9/volumes"
Sep 16 14:31:19.185603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.185588    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" path="/var/lib/kubelet/pods/50cbf182-f238-4b7b-8cd1-4baf4c56952b/volumes"
Sep 16 14:31:19.185893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:19.185881    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="be27ef0f-6e14-4b6d-a57e-d9013299f9b0" path="/var/lib/kubelet/pods/be27ef0f-6e14-4b6d-a57e-d9013299f9b0/volumes"
Sep 16 14:31:20.097846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.097819    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:20.131683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.131657    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca\") pod \"55854f11-049a-44fa-a617-1391555207f1\" (UID: \"55854f11-049a-44fa-a617-1391555207f1\") "
Sep 16 14:31:20.132002 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.131981    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "55854f11-049a-44fa-a617-1391555207f1" (UID: "55854f11-049a-44fa-a617-1391555207f1"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:20.233096 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.232971    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/55854f11-049a-44fa-a617-1391555207f1-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:20.973588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.973532    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"
Sep 16 14:31:20.973774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.973600    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs" event={"ID":"55854f11-049a-44fa-a617-1391555207f1","Type":"ContainerDied","Data":"797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281"}
Sep 16 14:31:20.973774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:20.973652    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="797e4e7308b6632d625a0dd9888652c087cda65cb0bd8a9157d32014be5d6281"
Sep 16 14:31:23.820030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:23.819969    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" podUID="18e1a6b5-fe93-4274-900e-579edec5830c" containerName="main" containerID="cri-o://812d2cedd4e113b638aa5f89809529806aee3a0c966dfd105fd819c92b40e1f2" gracePeriod=45
Sep 16 14:31:23.995354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:23.995320    2580 generic.go:356] "Generic (PLEG): container finished" podID="18e1a6b5-fe93-4274-900e-579edec5830c" containerID="812d2cedd4e113b638aa5f89809529806aee3a0c966dfd105fd819c92b40e1f2" exitCode=0
Sep 16 14:31:23.995513 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:23.995397    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" event={"ID":"18e1a6b5-fe93-4274-900e-579edec5830c","Type":"ContainerDied","Data":"812d2cedd4e113b638aa5f89809529806aee3a0c966dfd105fd819c92b40e1f2"}
Sep 16 14:31:24.075004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.074934    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:31:24.177592 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177549    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w97vt\" (UniqueName: \"kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.177785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.177785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177651    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.177785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.177785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177767    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.178020 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.177858    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm\") pod \"18e1a6b5-fe93-4274-900e-579edec5830c\" (UID: \"18e1a6b5-fe93-4274-900e-579edec5830c\") "
Sep 16 14:31:24.178441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.178395    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home" (OuterVolumeSpecName: "home") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.178578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.178495    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.178578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.178512    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache" (OuterVolumeSpecName: "model-cache") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.179995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.179951    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt" (OuterVolumeSpecName: "kube-api-access-w97vt") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "kube-api-access-w97vt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:24.180430 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.180407    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm" (OuterVolumeSpecName: "dshm") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:24.180512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.180478    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "18e1a6b5-fe93-4274-900e-579edec5830c" (UID: "18e1a6b5-fe93-4274-900e-579edec5830c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:31:24.278809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278777    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-dshm\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.278809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278810    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w97vt\" (UniqueName: \"kubernetes.io/projected/18e1a6b5-fe93-4274-900e-579edec5830c-kube-api-access-w97vt\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.278809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278826    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/18e1a6b5-fe93-4274-900e-579edec5830c-tls-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.279058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278838    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-tmp-dir\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.279058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278851    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-home\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:24.279058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:24.278863    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/18e1a6b5-fe93-4274-900e-579edec5830c-model-cache\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:25.001631 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.001600    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"
Sep 16 14:31:25.002033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.001631    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh" event={"ID":"18e1a6b5-fe93-4274-900e-579edec5830c","Type":"ContainerDied","Data":"63f85833181cbd7eebdb7940025befb5fe352d6f015e8ad0733db38e5fde24ef"}
Sep 16 14:31:25.002033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.001686    2580 scope.go:117] "RemoveContainer" containerID="812d2cedd4e113b638aa5f89809529806aee3a0c966dfd105fd819c92b40e1f2"
Sep 16 14:31:25.025373 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.025346    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:31:25.031851 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.031825    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-11636c/embedding-w0-11636c-kserve-5cbcfc56bb-4lrbh"]
Sep 16 14:31:25.184853 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:25.184821    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="18e1a6b5-fe93-4274-900e-579edec5830c" path="/var/lib/kubelet/pods/18e1a6b5-fe93-4274-900e-579edec5830c/volumes"
Sep 16 14:31:26.192420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:26.192380    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:31:26.192860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:26.192609    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy" containerID="cri-o://f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c" gracePeriod=30
Sep 16 14:31:28.002321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.002290    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:31:28.002825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.002547    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" podUID="894bad99-4db0-45ec-9c68-bc4be62f9d71" containerName="payload-processing" containerID="cri-o://ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e" gracePeriod=30
Sep 16 14:31:28.009093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.009068    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:31:28.009296 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.009274    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" podUID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" containerName="payload-pre-processing" containerID="cri-o://47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3" gracePeriod=30
Sep 16 14:31:28.281995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.281972    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:31:28.284683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.284664    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:31:28.322428 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.321310    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume\") pod \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") "
Sep 16 14:31:28.322428 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.321358    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whrt9\" (UniqueName: \"kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9\") pod \"894bad99-4db0-45ec-9c68-bc4be62f9d71\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") "
Sep 16 14:31:28.322428 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.321414    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume\") pod \"894bad99-4db0-45ec-9c68-bc4be62f9d71\" (UID: \"894bad99-4db0-45ec-9c68-bc4be62f9d71\") "
Sep 16 14:31:28.322428 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.321451    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zbpt7\" (UniqueName: \"kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7\") pod \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\" (UID: \"497286c6-d963-4d39-81ae-f5fafa8e0ad2\") "
Sep 16 14:31:28.323016 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.322893    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "497286c6-d963-4d39-81ae-f5fafa8e0ad2" (UID: "497286c6-d963-4d39-81ae-f5fafa8e0ad2"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:28.323200 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.323160    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "894bad99-4db0-45ec-9c68-bc4be62f9d71" (UID: "894bad99-4db0-45ec-9c68-bc4be62f9d71"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:28.331839 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.331792    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9" (OuterVolumeSpecName: "kube-api-access-whrt9") pod "894bad99-4db0-45ec-9c68-bc4be62f9d71" (UID: "894bad99-4db0-45ec-9c68-bc4be62f9d71"). InnerVolumeSpecName "kube-api-access-whrt9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:28.332141 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.332120    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7" (OuterVolumeSpecName: "kube-api-access-zbpt7") pod "497286c6-d963-4d39-81ae-f5fafa8e0ad2" (UID: "497286c6-d963-4d39-81ae-f5fafa8e0ad2"). InnerVolumeSpecName "kube-api-access-zbpt7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:28.373513 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.373479    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"]
Sep 16 14:31:28.380376 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.380347    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-2524a893-tbchs"]
Sep 16 14:31:28.423254 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.423215    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whrt9\" (UniqueName: \"kubernetes.io/projected/894bad99-4db0-45ec-9c68-bc4be62f9d71-kube-api-access-whrt9\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:28.423254 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.423247    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/894bad99-4db0-45ec-9c68-bc4be62f9d71-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:28.423254 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.423258    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zbpt7\" (UniqueName: \"kubernetes.io/projected/497286c6-d963-4d39-81ae-f5fafa8e0ad2-kube-api-access-zbpt7\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:28.423495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:28.423266    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/497286c6-d963-4d39-81ae-f5fafa8e0ad2-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:29.029357 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.029324    2580 generic.go:356] "Generic (PLEG): container finished" podID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" containerID="47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3" exitCode=0
Sep 16 14:31:29.029857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.029369    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"
Sep 16 14:31:29.029857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.029416    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" event={"ID":"497286c6-d963-4d39-81ae-f5fafa8e0ad2","Type":"ContainerDied","Data":"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"}
Sep 16 14:31:29.029857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.029477    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km" event={"ID":"497286c6-d963-4d39-81ae-f5fafa8e0ad2","Type":"ContainerDied","Data":"bc3ae0f8e9b22759a00fff84b1735407310c62f5c0f49ef783cab57bcc440947"}
Sep 16 14:31:29.029857 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.029512    2580 scope.go:117] "RemoveContainer" containerID="47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"
Sep 16 14:31:29.030898 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.030705    2580 generic.go:356] "Generic (PLEG): container finished" podID="894bad99-4db0-45ec-9c68-bc4be62f9d71" containerID="ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e" exitCode=0
Sep 16 14:31:29.030898 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.030758    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"
Sep 16 14:31:29.030898 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.030782    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" event={"ID":"894bad99-4db0-45ec-9c68-bc4be62f9d71","Type":"ContainerDied","Data":"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"}
Sep 16 14:31:29.030898 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.030836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb" event={"ID":"894bad99-4db0-45ec-9c68-bc4be62f9d71","Type":"ContainerDied","Data":"b628954342dc9a3ca11d957563f7651136111407d0ba83a5dee0a0b374a14c3b"}
Sep 16 14:31:29.038830 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.038812    2580 scope.go:117] "RemoveContainer" containerID="47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"
Sep 16 14:31:29.039095 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:29.039076    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3\": container with ID starting with 47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3 not found: ID does not exist" containerID="47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"
Sep 16 14:31:29.039180 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.039102    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3"} err="failed to get container status \"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3\": rpc error: code = NotFound desc = could not find container \"47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3\": container with ID starting with 47706b56d71ca0f6d3148203916d0b7d60114b5704799e6ae631f9833eb19da3 not found: ID does not exist"
Sep 16 14:31:29.039180 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.039117    2580 scope.go:117] "RemoveContainer" containerID="ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"
Sep 16 14:31:29.047829 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.047813    2580 scope.go:117] "RemoveContainer" containerID="ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"
Sep 16 14:31:29.048077 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:29.048061    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e\": container with ID starting with ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e not found: ID does not exist" containerID="ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"
Sep 16 14:31:29.048126 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.048082    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e"} err="failed to get container status \"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e\": rpc error: code = NotFound desc = could not find container \"ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e\": container with ID starting with ee49a54ee441ee67122f214c7033be3e2736c2a414c37bb49b481e778f393a5e not found: ID does not exist"
Sep 16 14:31:29.063896 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.063874    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:31:29.065762 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.065743    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-2524a893-5cbdcdf6b6-h64km"]
Sep 16 14:31:29.085447 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.085414    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:31:29.087204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.087181    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-mt-2524a893-594b9fd55c-zs9lb"]
Sep 16 14:31:29.184438 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.184405    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" path="/var/lib/kubelet/pods/497286c6-d963-4d39-81ae-f5fafa8e0ad2/volumes"
Sep 16 14:31:29.184801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.184787    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="55854f11-049a-44fa-a617-1391555207f1" path="/var/lib/kubelet/pods/55854f11-049a-44fa-a617-1391555207f1/volumes"
Sep 16 14:31:29.185131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:29.185119    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="894bad99-4db0-45ec-9c68-bc4be62f9d71" path="/var/lib/kubelet/pods/894bad99-4db0-45ec-9c68-bc4be62f9d71/volumes"
Sep 16 14:31:31.444809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.444785    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:31:31.553617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553515    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553576    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553644    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553722    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553753    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-945qv\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553788    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.553827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553819    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.554098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553856    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.554098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553884    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy\") pod \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\" (UID: \"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81\") "
Sep 16 14:31:31.554098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:31.554098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.553996    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:31.554302 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554110    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data" (OuterVolumeSpecName: "istio-data") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:31.554302 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554164    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:31.554302 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554211    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:31.554302 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554280    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.554508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554304    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.554508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554320    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.554508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554332    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.554508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.554344    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.555874 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.555844    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token" (OuterVolumeSpecName: "istio-token") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:31.555985 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.555901    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:31:31.556040 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.556017    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv" (OuterVolumeSpecName: "kube-api-access-945qv") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "kube-api-access-945qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:31.556116 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.556094    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" (UID: "0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:31.655352 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.655319    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-945qv\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-kube-api-access-945qv\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.655352 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.655347    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.655352 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.655357    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:31.655585 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:31.655365    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:32.055107 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.055066    2580 generic.go:356] "Generic (PLEG): container finished" podID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerID="f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c" exitCode=0
Sep 16 14:31:32.055289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.055115    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"
Sep 16 14:31:32.055289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.055158    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerDied","Data":"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"}
Sep 16 14:31:32.055289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.055216    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm" event={"ID":"0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81","Type":"ContainerDied","Data":"dd3c4ea62a95dffdbea659d5a755e0c078d4f3c3f1315a9260a75e13a3c63339"}
Sep 16 14:31:32.055289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.055241    2580 scope.go:117] "RemoveContainer" containerID="f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"
Sep 16 14:31:32.064948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.064930    2580 scope.go:117] "RemoveContainer" containerID="42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"
Sep 16 14:31:32.072786 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.072768    2580 scope.go:117] "RemoveContainer" containerID="f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"
Sep 16 14:31:32.073038 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:32.073019    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c\": container with ID starting with f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c not found: ID does not exist" containerID="f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"
Sep 16 14:31:32.073101 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.073045    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c"} err="failed to get container status \"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c\": rpc error: code = NotFound desc = could not find container \"f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c\": container with ID starting with f4fce6fc6ec600fad233d3fccd5ea6e7b98ea5981d85050e68218f053cf8921c not found: ID does not exist"
Sep 16 14:31:32.073101 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.073063    2580 scope.go:117] "RemoveContainer" containerID="42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"
Sep 16 14:31:32.073314 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:32.073300    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a\": container with ID starting with 42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a not found: ID does not exist" containerID="42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"
Sep 16 14:31:32.073363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.073320    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a"} err="failed to get container status \"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a\": rpc error: code = NotFound desc = could not find container \"42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a\": container with ID starting with 42ec4cf24ceb95e6770eb8e53758dea15eef6c9b6e5e47c6e96f8fed90d8ec6a not found: ID does not exist"
Sep 16 14:31:32.080464 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.080440    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:31:32.087578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:32.087545    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-11636c-openshift-default-6fdb7776b8-ltshm"]
Sep 16 14:31:33.184247 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:33.184198    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" path="/var/lib/kubelet/pods/0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81/volumes"
Sep 16 14:31:41.741948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:41.741910    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:31:41.742474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:41.742146    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" containerID="cri-o://0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983" gracePeriod=30
Sep 16 14:31:42.341404 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:42.341368    2580 patch_prober.go:28] interesting pod/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:31:42.341594 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:42.341427    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:31:46.989539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:46.989514    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:47.117284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117181    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117229    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q5wfb\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117284 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117272    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117315    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117353    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117416    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117462    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117496    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs\") pod \"5c35a036-770d-40a3-b07e-8dac463f1052\" (UID: \"5c35a036-770d-40a3-b07e-8dac463f1052\") "
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117699    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117705    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117740    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data" (OuterVolumeSpecName: "istio-data") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117902    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117911    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117932    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.117975 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117947    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.118354 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.117985    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:31:47.119609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.119578    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token" (OuterVolumeSpecName: "istio-token") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:47.119734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.119605    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:31:47.119734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.119683    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:31:47.119734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.119689    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb" (OuterVolumeSpecName: "kube-api-access-q5wfb") pod "5c35a036-770d-40a3-b07e-8dac463f1052" (UID: "5c35a036-770d-40a3-b07e-8dac463f1052"). InnerVolumeSpecName "kube-api-access-q5wfb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:31:47.135998 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.135970    2580 generic.go:356] "Generic (PLEG): container finished" podID="5c35a036-770d-40a3-b07e-8dac463f1052" containerID="0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983" exitCode=0
Sep 16 14:31:47.136138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.136034    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"
Sep 16 14:31:47.136138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.136050    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerDied","Data":"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"}
Sep 16 14:31:47.136138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.136112    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz" event={"ID":"5c35a036-770d-40a3-b07e-8dac463f1052","Type":"ContainerDied","Data":"aa2b6056170f0b3e958da6333b00f700e5fbf0a5a401949071feebed32c73736"}
Sep 16 14:31:47.136138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.136135    2580 scope.go:117] "RemoveContainer" containerID="0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"
Sep 16 14:31:47.145072 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.145057    2580 scope.go:117] "RemoveContainer" containerID="eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"
Sep 16 14:31:47.152615 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.152565    2580 scope.go:117] "RemoveContainer" containerID="0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"
Sep 16 14:31:47.152877 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:47.152856    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983\": container with ID starting with 0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983 not found: ID does not exist" containerID="0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"
Sep 16 14:31:47.152941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.152888    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983"} err="failed to get container status \"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983\": rpc error: code = NotFound desc = could not find container \"0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983\": container with ID starting with 0991a792180eb5644b49668fa1cb7dc8a3f30d562412ed42ef46a164f00f3983 not found: ID does not exist"
Sep 16 14:31:47.152941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.152913    2580 scope.go:117] "RemoveContainer" containerID="eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"
Sep 16 14:31:47.153223 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:31:47.153202    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6\": container with ID starting with eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6 not found: ID does not exist" containerID="eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"
Sep 16 14:31:47.153304 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.153231    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6"} err="failed to get container status \"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6\": rpc error: code = NotFound desc = could not find container \"eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6\": container with ID starting with eff97e7a300513e7159c615547856b6f1e3a38cab9261f2ce6a901d499938be6 not found: ID does not exist"
Sep 16 14:31:47.185496 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.185469    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:31:47.208492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.208462    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-2524a893-openshift-default-7b75ffcb48-rxndz"]
Sep 16 14:31:47.218836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218813    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5c35a036-770d-40a3-b07e-8dac463f1052-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.218836 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218835    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.218963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218843    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.218963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218852    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q5wfb\" (UniqueName: \"kubernetes.io/projected/5c35a036-770d-40a3-b07e-8dac463f1052-kube-api-access-q5wfb\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.218963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218861    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5c35a036-770d-40a3-b07e-8dac463f1052-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:47.218963 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:47.218870    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5c35a036-770d-40a3-b07e-8dac463f1052-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:31:49.181995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:49.181965    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:31:49.185000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:31:49.184975    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" path="/var/lib/kubelet/pods/5c35a036-770d-40a3-b07e-8dac463f1052/volumes"
Sep 16 14:32:07.860174 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860137    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860518    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860531    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860545    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860551    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860571    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" containerName="payload-pre-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860577    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" containerName="payload-pre-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860588    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="894bad99-4db0-45ec-9c68-bc4be62f9d71" containerName="payload-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860593    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="894bad99-4db0-45ec-9c68-bc4be62f9d71" containerName="payload-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860600    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860606    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860619    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3920cc91-0b37-4141-bf1f-fab93c9977b9" containerName="payload-pre-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860626    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3920cc91-0b37-4141-bf1f-fab93c9977b9" containerName="payload-pre-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860635    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="18e1a6b5-fe93-4274-900e-579edec5830c" containerName="main"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860639    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="18e1a6b5-fe93-4274-900e-579edec5830c" containerName="main"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860651    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55854f11-049a-44fa-a617-1391555207f1" containerName="revoke-keys"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860656    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="55854f11-049a-44fa-a617-1391555207f1" containerName="revoke-keys"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860665    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" containerName="payload-processing"
Sep 16 14:32:07.860701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860670    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" containerName="payload-processing"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860728    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="894bad99-4db0-45ec-9c68-bc4be62f9d71" containerName="payload-processing"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860737    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860746    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="55854f11-049a-44fa-a617-1391555207f1" containerName="revoke-keys"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860753    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="18e1a6b5-fe93-4274-900e-579edec5830c" containerName="main"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860758    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="497286c6-d963-4d39-81ae-f5fafa8e0ad2" containerName="payload-pre-processing"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860764    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860771    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="50cbf182-f238-4b7b-8cd1-4baf4c56952b" containerName="payload-processing"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860779    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3920cc91-0b37-4141-bf1f-fab93c9977b9" containerName="payload-pre-processing"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860787    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.861271 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.860793    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5c35a036-770d-40a3-b07e-8dac463f1052" containerName="istio-proxy"
Sep 16 14:32:07.865367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.865343    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:07.869919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.869894    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-dc3e5b-openshift-default-dockercfg-szz6q\""
Sep 16 14:32:07.974378 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:07.974348    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:32:08.015320 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015288    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015332    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015359    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015441    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015487    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015523    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015581    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015627    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.015726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.015645    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116380    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116437 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116425    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116462    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116494    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116518    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116594    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.116768 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116635    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.117091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.116995    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.117091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.117053    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.117091 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.117074    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.117259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.117156    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.117259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.117215    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.118761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.118743    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.119054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.119037    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.128368 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.128337    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.134210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.134185    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token\") pod \"e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") " pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.175828 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.175799    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:08.478639 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.478601    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:32:08.482322 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:08.482293    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd96c0af5_b2f9_44a6_9853_0ccf1ead036f.slice/crio-0e4cc27700f92c0b1a977a30952dcebcfbf309b78c65959556d3f9c8eff4626a WatchSource:0}: Error finding container 0e4cc27700f92c0b1a977a30952dcebcfbf309b78c65959556d3f9c8eff4626a: Status 404 returned error can't find the container with id 0e4cc27700f92c0b1a977a30952dcebcfbf309b78c65959556d3f9c8eff4626a
Sep 16 14:32:08.484399 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.484370    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:08.484457 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.484433    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:08.484492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:08.484460    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:09.259533 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:09.259500    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerStarted","Data":"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"}
Sep 16 14:32:09.259942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:09.259569    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerStarted","Data":"0e4cc27700f92c0b1a977a30952dcebcfbf309b78c65959556d3f9c8eff4626a"}
Sep 16 14:32:09.303703 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:09.303652    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podStartSLOduration=2.30363493 podStartE2EDuration="2.30363493s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:09.302325256 +0000 UTC m=+1450.708905464" watchObservedRunningTime="2026-09-16 14:32:09.30363493 +0000 UTC m=+1450.710215138"
Sep 16 14:32:10.176641 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:10.176596    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:10.181795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:10.181769    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:10.266071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:10.266034    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:10.266917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:10.266894    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:17.315817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.315782    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:32:17.316241 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.316140    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:17.316241 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.316151    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e76de17-1ea5-4e05-a7e5-b8b1d9f40b81" containerName="istio-proxy"
Sep 16 14:32:17.319381 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.319362    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.324016 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:32:17.323971    2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" is forbidden: User \"system:node:ip-10-0-138-80.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-80.ec2.internal' and this object" podUID="906246a1-0c08-4a64-aad3-bea251617860" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.324134 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:32:17.324069    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.ConfigMap: configmaps \"payload-processing-plugins-e2e-shared-a-w1-dc3e5b\" is forbidden: User \"system:node:ip-10-0-138-80.ec2.internal\" cannot list resource \"configmaps\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-dc3e5b\"" type="*v1.ConfigMap"
Sep 16 14:32:17.324181 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:32:17.324141    2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"payload-processing-e2e-shared-a-w1-dc3e5b-dockercfg-znvxq\" is forbidden: User \"system:node:ip-10-0-138-80.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-138-80.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-dc3e5b-dockercfg-znvxq\"" type="*v1.Secret"
Sep 16 14:32:17.340599 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.340538    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:32:17.417533 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.417494    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.417737 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.417604    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsm74\" (UniqueName: \"kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.425950 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.425917    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:32:17.429326 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.429310    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:17.478227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.478187    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:32:17.518731 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.518697    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsm74\" (UniqueName: \"kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.518876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.518784    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.533087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.533057    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsm74\" (UniqueName: \"kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:17.620195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.620096    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:17.620195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.620143    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5pxj\" (UniqueName: \"kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:17.720879 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.720838    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:17.721072 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.720897    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5pxj\" (UniqueName: \"kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:17.737749 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:17.737712    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5pxj\" (UniqueName: \"kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:18.174107 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.174075    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-dc3e5b\""
Sep 16 14:32:18.180004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.179972    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:18.182059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.182036    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:18.207920 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.207897    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-dc3e5b-dockercfg-znvxq\""
Sep 16 14:32:18.229968 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.229935    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:18.338910 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.338879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:18.527971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.527945    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:32:18.530631 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:18.530607    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906246a1_0c08_4a64_aad3_bea251617860.slice/crio-4ca6c86cfa7f7e8a23daf62220323cb59e616bae3f5b9f3fe28d9a46adbaea17 WatchSource:0}: Error finding container 4ca6c86cfa7f7e8a23daf62220323cb59e616bae3f5b9f3fe28d9a46adbaea17: Status 404 returned error can't find the container with id 4ca6c86cfa7f7e8a23daf62220323cb59e616bae3f5b9f3fe28d9a46adbaea17
Sep 16 14:32:18.638625 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:18.638589    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:32:18.642511 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:18.642471    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc69b163d_6eea_4a02_b9c0_5c995d907840.slice/crio-707631d52920b66b99b880fe0a14a50619ade7a25f4912edfa98105eb6bbbf06 WatchSource:0}: Error finding container 707631d52920b66b99b880fe0a14a50619ade7a25f4912edfa98105eb6bbbf06: Status 404 returned error can't find the container with id 707631d52920b66b99b880fe0a14a50619ade7a25f4912edfa98105eb6bbbf06
Sep 16 14:32:19.314329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.314298    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" event={"ID":"c69b163d-6eea-4a02-b9c0-5c995d907840","Type":"ContainerStarted","Data":"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"}
Sep 16 14:32:19.314539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.314353    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" event={"ID":"c69b163d-6eea-4a02-b9c0-5c995d907840","Type":"ContainerStarted","Data":"707631d52920b66b99b880fe0a14a50619ade7a25f4912edfa98105eb6bbbf06"}
Sep 16 14:32:19.314539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.314436    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:19.315863 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.315840    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" event={"ID":"906246a1-0c08-4a64-aad3-bea251617860","Type":"ContainerStarted","Data":"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"}
Sep 16 14:32:19.315967 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.315878    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" event={"ID":"906246a1-0c08-4a64-aad3-bea251617860","Type":"ContainerStarted","Data":"4ca6c86cfa7f7e8a23daf62220323cb59e616bae3f5b9f3fe28d9a46adbaea17"}
Sep 16 14:32:19.315967 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.315909    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:19.390702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.390652    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" podStartSLOduration=2.390634004 podStartE2EDuration="2.390634004s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:19.389124555 +0000 UTC m=+1460.795704763" watchObservedRunningTime="2026-09-16 14:32:19.390634004 +0000 UTC m=+1460.797214211"
Sep 16 14:32:19.392297 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:19.392260    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" podStartSLOduration=2.392247078 podStartE2EDuration="2.392247078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:19.355624509 +0000 UTC m=+1460.762204717" watchObservedRunningTime="2026-09-16 14:32:19.392247078 +0000 UTC m=+1460.798827330"
Sep 16 14:32:30.325683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:30.325647    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:32:30.326184 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:30.325735    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:32:31.382062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.381961    2580 generic.go:356] "Generic (PLEG): container finished" podID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerID="b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a" exitCode=0
Sep 16 14:32:31.382062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.382019    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerDied","Data":"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"}
Sep 16 14:32:31.382510 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.382400    2580 scope.go:117] "RemoveContainer" containerID="b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"
Sep 16 14:32:31.383454 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.383424    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:31.383520 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.383506    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:31.383572 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:31.383542    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:32.392785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:32.392739    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerStarted","Data":"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"}
Sep 16 14:32:33.177072 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:33.177028    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:33.177278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:33.177161    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 16 14:32:33.177278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:33.177214    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 16 14:32:34.176579 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:34.176515    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 16 14:32:34.177077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:34.176622    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 16 14:32:35.184819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:35.183233    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 16 14:32:35.184819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:35.183297    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 16 14:32:36.176722 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:36.176675    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 16 14:32:36.176947 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:36.176749    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 16 14:32:37.176538 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:37.176500    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused" start-of-body=
Sep 16 14:32:37.177170 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:37.177136    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.60:15021/healthz/ready\": dial tcp 10.133.0.60:15021: connect: connection refused"
Sep 16 14:32:38.181569 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:38.181531    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:38.181995 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:38.181823    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:38.182478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:38.182457    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:32:42.388355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.388323    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:32:42.391154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.391132    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.394347 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.394330    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-bdfd4a-openshift-default-dockercfg-79kj6\""
Sep 16 14:32:42.476036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.475996    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476051    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zfpfq\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476206    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476373 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476296    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476459    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476549 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476515    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476628    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.476717 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.476674    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.494188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.494161    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:32:42.578071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578030    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578092    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578117    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578165    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578259 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zfpfq\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578469 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578381    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578804    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.578958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578934    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.578979    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579094 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.579030    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579094 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.579078    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.579193    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579270 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.579215    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.579524 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.579503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.580644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.580625    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.580866 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.580848    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.591198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.591177    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.591458 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.591436    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zfpfq\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq\") pod \"e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") " pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.700886 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.700855    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:42.980946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.980917    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:32:42.982676 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:42.982647    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e80ecc0_9e86_4f16_8854_280fce70bc76.slice/crio-ecc1cd30dd4cc4cb82a16fa5c17599c0aa57edc0408a63c2996a6b1558f2d09e WatchSource:0}: Error finding container ecc1cd30dd4cc4cb82a16fa5c17599c0aa57edc0408a63c2996a6b1558f2d09e: Status 404 returned error can't find the container with id ecc1cd30dd4cc4cb82a16fa5c17599c0aa57edc0408a63c2996a6b1558f2d09e
Sep 16 14:32:42.984848 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.984814    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:42.984966 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.984877    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:42.984966 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:42.984904    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:32:43.448036 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:43.447999    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerStarted","Data":"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"}
Sep 16 14:32:43.448576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:43.448048    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerStarted","Data":"ecc1cd30dd4cc4cb82a16fa5c17599c0aa57edc0408a63c2996a6b1558f2d09e"}
Sep 16 14:32:43.472143 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:43.472083    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podStartSLOduration=1.47206309 podStartE2EDuration="1.47206309s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:43.471740821 +0000 UTC m=+1484.878321023" watchObservedRunningTime="2026-09-16 14:32:43.47206309 +0000 UTC m=+1484.878643299"
Sep 16 14:32:43.701650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:43.701536    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:44.706677 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:44.706646    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:45.458997 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:45.458965    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:45.459880 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:45.459862    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:32:49.170345 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.170314    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:32:49.175911 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.175878    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.179685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.179657    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w1-bdfd4a\""
Sep 16 14:32:49.179819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.179801    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w1-bdfd4a-dockercfg-bcr9k\""
Sep 16 14:32:49.188138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.188113    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:32:49.239783 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.239758    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:32:49.242234 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.242217    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.250882 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.250854    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c44p9\" (UniqueName: \"kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.250980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.250903    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.251053 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.251029    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7tbd\" (UniqueName: \"kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.251181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.251157    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.255526 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.255501    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:32:49.351576 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.351534    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7tbd\" (UniqueName: \"kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.351772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.351634    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.351772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.351674    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c44p9\" (UniqueName: \"kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.351772 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.351695    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.352327 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.352306    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.352407 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.352336    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.365926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.365891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c44p9\" (UniqueName: \"kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9\") pod \"payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") " pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.366045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.365891    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7tbd\" (UniqueName: \"kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd\") pod \"payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.489169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.489146    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:49.552168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.552048    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:49.768734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.768707    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:32:49.770916 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:49.770890    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod702d5aa7_e543_4d31_9304_c60c06a9dfbf.slice/crio-4a5dbb3e7a74ad93cb30b3042d2df71a5cc18be65de73d24d977ff557b5b5f7b WatchSource:0}: Error finding container 4a5dbb3e7a74ad93cb30b3042d2df71a5cc18be65de73d24d977ff557b5b5f7b: Status 404 returned error can't find the container with id 4a5dbb3e7a74ad93cb30b3042d2df71a5cc18be65de73d24d977ff557b5b5f7b
Sep 16 14:32:49.848854 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:49.848824    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:32:49.852759 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:32:49.852732    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podabf20d47_3f33_4a67_afb7_4ccdd7540cbf.slice/crio-9be53b5026be129efde1ee38a14e5d4828c256ab7d12a8de18e843c7478bba7e WatchSource:0}: Error finding container 9be53b5026be129efde1ee38a14e5d4828c256ab7d12a8de18e843c7478bba7e: Status 404 returned error can't find the container with id 9be53b5026be129efde1ee38a14e5d4828c256ab7d12a8de18e843c7478bba7e
Sep 16 14:32:50.486852 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.486811    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" event={"ID":"abf20d47-3f33-4a67-afb7-4ccdd7540cbf","Type":"ContainerStarted","Data":"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"}
Sep 16 14:32:50.487359 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.486889    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" event={"ID":"abf20d47-3f33-4a67-afb7-4ccdd7540cbf","Type":"ContainerStarted","Data":"9be53b5026be129efde1ee38a14e5d4828c256ab7d12a8de18e843c7478bba7e"}
Sep 16 14:32:50.487359 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.486987    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:32:50.488727 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.488704    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" event={"ID":"702d5aa7-e543-4d31-9304-c60c06a9dfbf","Type":"ContainerStarted","Data":"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"}
Sep 16 14:32:50.488847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.488749    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" event={"ID":"702d5aa7-e543-4d31-9304-c60c06a9dfbf","Type":"ContainerStarted","Data":"4a5dbb3e7a74ad93cb30b3042d2df71a5cc18be65de73d24d977ff557b5b5f7b"}
Sep 16 14:32:50.488847 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.488776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:32:50.534097 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.534046    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" podStartSLOduration=1.5340292930000001 podStartE2EDuration="1.534029293s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:50.53379285 +0000 UTC m=+1491.940373062" watchObservedRunningTime="2026-09-16 14:32:50.534029293 +0000 UTC m=+1491.940609499"
Sep 16 14:32:50.570888 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:50.570836    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" podStartSLOduration=1.5708220160000002 podStartE2EDuration="1.570822016s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:32:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:32:50.569072507 +0000 UTC m=+1491.975652737" watchObservedRunningTime="2026-09-16 14:32:50.570822016 +0000 UTC m=+1491.977402223"
Sep 16 14:32:59.244984 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:32:59.244952    2580 scope.go:117] "RemoveContainer" containerID="b170e521728ba64afdfdaee230d965d611420ee0d47b5097ff26057c225e3b51"
Sep 16 14:33:01.499093 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:01.499062    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:33:01.499583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:01.499382    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:33:02.949867 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:02.949831    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:33:05.571745 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.571665    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerID="094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf" exitCode=0
Sep 16 14:33:05.572159 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.571738    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerDied","Data":"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"}
Sep 16 14:33:05.572159 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.572153    2580 scope.go:117] "RemoveContainer" containerID="094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"
Sep 16 14:33:05.573230 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.573202    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:05.573295 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.573281    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:05.573334 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:05.573315    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:06.596035 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:06.595996    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerStarted","Data":"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"}
Sep 16 14:33:06.701022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:06.700986    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:33:06.701219 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:06.701191    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 16 14:33:06.701272 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:06.701256    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 16 14:33:07.701531 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:07.701484    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 16 14:33:07.702020 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:07.701585    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 16 14:33:08.701957 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:08.701923    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 16 14:33:08.702333 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:08.701977    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 16 14:33:09.701897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:09.701857    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 16 14:33:09.702077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:09.701917    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 16 14:33:10.701891 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:10.701857    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused" start-of-body=
Sep 16 14:33:10.702069 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:10.701911    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.63:15021/healthz/ready\": dial tcp 10.133.0.63:15021: connect: connection refused"
Sep 16 14:33:11.181656 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:11.181550    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:33:11.781353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:11.781321    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:33:11.781630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:11.781601    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:33:11.782983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:11.782960    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:33:17.972286 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:17.972238    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" podUID="3ba1bb49-4fd4-4893-b355-740e404748d6" containerName="main" containerID="cri-o://9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f" gracePeriod=45
Sep 16 14:33:18.214710 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.214683    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:33:18.280064 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.279977    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280064 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280043    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5rrxx\" (UniqueName: \"kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280069    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280136    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280211    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280385 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280258    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm\") pod \"3ba1bb49-4fd4-4893-b355-740e404748d6\" (UID: \"3ba1bb49-4fd4-4893-b355-740e404748d6\") "
Sep 16 14:33:18.280385 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280267    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:18.280385 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280364    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:18.280550 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280415    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home" (OuterVolumeSpecName: "home") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:18.280648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280551    2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-home\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.280648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280595    2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-model-cache\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.280648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.280605    2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-tmp-dir\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.282321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.282291    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm" (OuterVolumeSpecName: "dshm") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:33:18.282431 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.282348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 16 14:33:18.282431 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.282387    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx" (OuterVolumeSpecName: "kube-api-access-5rrxx") pod "3ba1bb49-4fd4-4893-b355-740e404748d6" (UID: "3ba1bb49-4fd4-4893-b355-740e404748d6"). InnerVolumeSpecName "kube-api-access-5rrxx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:33:18.381684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.381652    2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/3ba1bb49-4fd4-4893-b355-740e404748d6-tls-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.381684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.381679    2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/3ba1bb49-4fd4-4893-b355-740e404748d6-dshm\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.381856 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.381693    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5rrxx\" (UniqueName: \"kubernetes.io/projected/3ba1bb49-4fd4-4893-b355-740e404748d6-kube-api-access-5rrxx\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:33:18.658035 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.657953    2580 generic.go:356] "Generic (PLEG): container finished" podID="3ba1bb49-4fd4-4893-b355-740e404748d6" containerID="9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f" exitCode=0
Sep 16 14:33:18.658035 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.657993    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"
Sep 16 14:33:18.658236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.657988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" event={"ID":"3ba1bb49-4fd4-4893-b355-740e404748d6","Type":"ContainerDied","Data":"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"}
Sep 16 14:33:18.658236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.658101    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79" event={"ID":"3ba1bb49-4fd4-4893-b355-740e404748d6","Type":"ContainerDied","Data":"c792198b20567a9dada5d5b21b7e931ba2da4ba265ce3db94a57845ec083673a"}
Sep 16 14:33:18.658236 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.658119    2580 scope.go:117] "RemoveContainer" containerID="9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"
Sep 16 14:33:18.667332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.667314    2580 scope.go:117] "RemoveContainer" containerID="9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"
Sep 16 14:33:18.667652 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:33:18.667631    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f\": container with ID starting with 9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f not found: ID does not exist" containerID="9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"
Sep 16 14:33:18.667708 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.667661    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f"} err="failed to get container status \"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f\": rpc error: code = NotFound desc = could not find container \"9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f\": container with ID starting with 9f83866858c3434b5847386caa9071b9af17a960b817a50b59da3effd531461f not found: ID does not exist"
Sep 16 14:33:18.694139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.694117    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:33:18.702893 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:18.702873    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w2-d31d7c/auth-test-model-d31d7c-kserve-6d6d495995-mjq79"]
Sep 16 14:33:19.184527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:19.184496    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ba1bb49-4fd4-4893-b355-740e404748d6" path="/var/lib/kubelet/pods/3ba1bb49-4fd4-4893-b355-740e404748d6/volumes"
Sep 16 14:33:25.403000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.402925    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:33:25.403346 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.403314    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ba1bb49-4fd4-4893-b355-740e404748d6" containerName="main"
Sep 16 14:33:25.403346 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.403325    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ba1bb49-4fd4-4893-b355-740e404748d6" containerName="main"
Sep 16 14:33:25.403423 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.403397    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="3ba1bb49-4fd4-4893-b355-740e404748d6" containerName="main"
Sep 16 14:33:25.406169 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.406151    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.409282 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.409264    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-e9829ef2-openshift-default-dockercfg-zw95f\""
Sep 16 14:33:25.427199 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.427166    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:33:25.450499 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450463    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f54f\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450514    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450533    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450591    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450622    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450711 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450649    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450729    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450782    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.450959 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.450821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.551978    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552054    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f54f\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552099    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552127    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552161    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552204    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552241    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552298    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552336    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552418    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552503    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552662    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552765 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552751    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.552964 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.552944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.554609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.554588    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.554913 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.554896    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.561205 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.561175    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f54f\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.562988 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.562968    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token\") pod \"e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") " pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.716096 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.716061    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:25.994440 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.994404    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:33:25.996106 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:33:25.996080    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6443ca3_08cd_44d5_b0d2_97eb522cc085.slice/crio-7adaac2433c6a553adaafaa4c9b23963e5072b6cac1932c78c9c4d2b27fd2d19 WatchSource:0}: Error finding container 7adaac2433c6a553adaafaa4c9b23963e5072b6cac1932c78c9c4d2b27fd2d19: Status 404 returned error can't find the container with id 7adaac2433c6a553adaafaa4c9b23963e5072b6cac1932c78c9c4d2b27fd2d19
Sep 16 14:33:25.998316 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.998287    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:25.998401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.998351    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:25.998401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:25.998380    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:26.707422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:26.707384    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerStarted","Data":"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"}
Sep 16 14:33:26.707827 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:26.707457    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerStarted","Data":"7adaac2433c6a553adaafaa4c9b23963e5072b6cac1932c78c9c4d2b27fd2d19"}
Sep 16 14:33:26.716972 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:26.716940    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:26.730442 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:26.730397    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podStartSLOduration=1.730382536 podStartE2EDuration="1.730382536s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:33:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:33:26.729435505 +0000 UTC m=+1528.136015713" watchObservedRunningTime="2026-09-16 14:33:26.730382536 +0000 UTC m=+1528.136962758"
Sep 16 14:33:27.721687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:27.721659    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:28.719052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:28.719023    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:28.720077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:28.720058    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:50.839197 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.839165    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerID="0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0" exitCode=0
Sep 16 14:33:50.839696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.839241    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerDied","Data":"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"}
Sep 16 14:33:50.839696 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.839674    2580 scope.go:117] "RemoveContainer" containerID="0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"
Sep 16 14:33:50.840859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.840835    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:50.840972 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.840904    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:50.840972 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:50.840935    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:33:51.846058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:51.846021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerStarted","Data":"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"}
Sep 16 14:33:52.717039 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:52.717001    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:52.717264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:52.717240    2580 patch_prober.go:28] interesting pod/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 14:33:52.717353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:52.717301    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 14:33:53.717142 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:53.717105    2580 patch_prober.go:28] interesting pod/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 14:33:53.717583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:53.717160    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 14:33:54.717343 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:54.717307    2580 patch_prober.go:28] interesting pod/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 14:33:54.717771 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:54.717378    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 14:33:55.717030 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:55.716998    2580 patch_prober.go:28] interesting pod/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 14:33:55.717218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:55.717067    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 14:33:56.716667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:56.716630    2580 patch_prober.go:28] interesting pod/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused" start-of-body=
Sep 16 14:33:56.717081 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:56.716706    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.66:15021/healthz/ready\": dial tcp 10.133.0.66:15021: connect: connection refused"
Sep 16 14:33:57.720649 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:57.720616    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:57.721102 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:57.720957    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:33:57.721637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:33:57.721619    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:34:23.178755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:34:23.178715    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:34:59.277100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:34:59.277024    2580 scope.go:117] "RemoveContainer" containerID="d2fe66b1ff87059b975f803241c6a72e2edcbfd7d06d699c5e8dd1dd7dd8891a"
Sep 16 14:35:34.262072 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.262032    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"]
Sep 16 14:35:34.264584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.264567    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:34.266762 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.266741    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:35:34.266897 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.266743    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:35:34.273730 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.273705    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"]
Sep 16 14:35:34.350938 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.350892    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz\" (UID: \"aa9c5739-127d-42e6-8190-bbd10a59e43e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:34.452250 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.452211    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz\" (UID: \"aa9c5739-127d-42e6-8190-bbd10a59e43e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:34.452862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.452844    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz\" (UID: \"aa9c5739-127d-42e6-8190-bbd10a59e43e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:34.577205 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.577092    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:34.850589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:34.850488    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"]
Sep 16 14:35:34.853725 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:35:34.853694    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa9c5739_127d_42e6_8190_bbd10a59e43e.slice/crio-3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9 WatchSource:0}: Error finding container 3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9: Status 404 returned error can't find the container with id 3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9
Sep 16 14:35:35.384819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:35.384784    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz" event={"ID":"aa9c5739-127d-42e6-8190-bbd10a59e43e","Type":"ContainerStarted","Data":"92c6f8d537832d22fce87cd67caaa2583127eab9932b08e5552261eedcfe8f90"}
Sep 16 14:35:35.385234 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:35.384833    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz" event={"ID":"aa9c5739-127d-42e6-8190-bbd10a59e43e","Type":"ContainerStarted","Data":"3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9"}
Sep 16 14:35:35.403031 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:35.402973    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz" podStartSLOduration=1.402953192 podStartE2EDuration="1.402953192s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:35:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:35:35.401925101 +0000 UTC m=+1656.808505309" watchObservedRunningTime="2026-09-16 14:35:35.402953192 +0000 UTC m=+1656.809533401"
Sep 16 14:35:36.391679 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:36.391644    2580 generic.go:356] "Generic (PLEG): container finished" podID="aa9c5739-127d-42e6-8190-bbd10a59e43e" containerID="92c6f8d537832d22fce87cd67caaa2583127eab9932b08e5552261eedcfe8f90" exitCode=0
Sep 16 14:35:36.392077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:36.391726    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz" event={"ID":"aa9c5739-127d-42e6-8190-bbd10a59e43e","Type":"ContainerDied","Data":"92c6f8d537832d22fce87cd67caaa2583127eab9932b08e5552261eedcfe8f90"}
Sep 16 14:35:37.545262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:37.545240    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:37.582774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:37.582681    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca\") pod \"aa9c5739-127d-42e6-8190-bbd10a59e43e\" (UID: \"aa9c5739-127d-42e6-8190-bbd10a59e43e\") "
Sep 16 14:35:37.582993 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:37.582972    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "aa9c5739-127d-42e6-8190-bbd10a59e43e" (UID: "aa9c5739-127d-42e6-8190-bbd10a59e43e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:35:37.583131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:37.583116    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/aa9c5739-127d-42e6-8190-bbd10a59e43e-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:35:38.405300 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:38.405272    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"
Sep 16 14:35:38.405458 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:38.405272    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz" event={"ID":"aa9c5739-127d-42e6-8190-bbd10a59e43e","Type":"ContainerDied","Data":"3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9"}
Sep 16 14:35:38.405458 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:38.405394    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f14da30e0d13033164deffc622fd59979f4a561bd6686f3a4883da6eca0f0c9"
Sep 16 14:35:44.179017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:44.178987    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:35:50.314388 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:50.314353    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"]
Sep 16 14:35:50.318911 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:50.318886    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-e9829ef2-v6zcz"]
Sep 16 14:35:51.184664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:35:51.184633    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa9c5739-127d-42e6-8190-bbd10a59e43e" path="/var/lib/kubelet/pods/aa9c5739-127d-42e6-8190-bbd10a59e43e/volumes"
Sep 16 14:36:59.503824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:36:59.503012    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:36:59.503824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:36:59.503338    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy" containerID="cri-o://60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1" gracePeriod=30
Sep 16 14:37:04.178928 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.178888    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:37:04.764401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.764371    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:37:04.826917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.826829    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.826917 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.826877    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.826927    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7f54f\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.826966    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827106    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:04.827172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827185    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data" (OuterVolumeSpecName: "istio-data") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:04.827351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827199    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827277    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827317    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827396    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert\") pod \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\" (UID: \"d6443ca3-08cd-44d5-b0d2-97eb522cc085\") "
Sep 16 14:37:04.827730 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827409    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:04.827795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827597    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:04.827795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827744    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:37:04.827907 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827884    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.828033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827906    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.828033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827921    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.828033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827934    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.828033 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.827947    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.829276 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.829242    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f" (OuterVolumeSpecName: "kube-api-access-7f54f") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "kube-api-access-7f54f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:04.829398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.829302    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:37:04.829398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.829306    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token" (OuterVolumeSpecName: "istio-token") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:37:04.829398 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.829363    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d6443ca3-08cd-44d5-b0d2-97eb522cc085" (UID: "d6443ca3-08cd-44d5-b0d2-97eb522cc085"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:37:04.844876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.844851    2580 generic.go:356] "Generic (PLEG): container finished" podID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerID="60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1" exitCode=0
Sep 16 14:37:04.844982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.844897    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"
Sep 16 14:37:04.844982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.844906    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerDied","Data":"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"}
Sep 16 14:37:04.844982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.844966    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g" event={"ID":"d6443ca3-08cd-44d5-b0d2-97eb522cc085","Type":"ContainerDied","Data":"7adaac2433c6a553adaafaa4c9b23963e5072b6cac1932c78c9c4d2b27fd2d19"}
Sep 16 14:37:04.845085 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.844991    2580 scope.go:117] "RemoveContainer" containerID="60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"
Sep 16 14:37:04.856467 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.856328    2580 scope.go:117] "RemoveContainer" containerID="0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"
Sep 16 14:37:04.864814 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.864797    2580 scope.go:117] "RemoveContainer" containerID="60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"
Sep 16 14:37:04.865087 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:37:04.865069    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1\": container with ID starting with 60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1 not found: ID does not exist" containerID="60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"
Sep 16 14:37:04.865127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.865098    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1"} err="failed to get container status \"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1\": rpc error: code = NotFound desc = could not find container \"60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1\": container with ID starting with 60ca08e208fe7297dde6833b93612d3172a37f4e54e3bcacdf5103b9cfe8dbd1 not found: ID does not exist"
Sep 16 14:37:04.865127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.865115    2580 scope.go:117] "RemoveContainer" containerID="0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"
Sep 16 14:37:04.865357 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:37:04.865341    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0\": container with ID starting with 0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0 not found: ID does not exist" containerID="0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"
Sep 16 14:37:04.865397 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.865364    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0"} err="failed to get container status \"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0\": rpc error: code = NotFound desc = could not find container \"0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0\": container with ID starting with 0a27078825342ad8f3bdb864d7a774ee49eefe25776c17ef0bb38f77bfa5a7c0 not found: ID does not exist"
Sep 16 14:37:04.870355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.870333    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:37:04.874726 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.874704    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-e9829ef2-openshift-default-596f4f7844-ssd8g"]
Sep 16 14:37:04.929419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.929389    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7f54f\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-kube-api-access-7f54f\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.929419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.929417    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.929626 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.929428    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:04.929626 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:04.929436    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d6443ca3-08cd-44d5-b0d2-97eb522cc085-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:37:05.184257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:05.184178    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" path="/var/lib/kubelet/pods/d6443ca3-08cd-44d5-b0d2-97eb522cc085/volumes"
Sep 16 14:37:44.292890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.292858    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293289    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9c5739-127d-42e6-8190-bbd10a59e43e" containerName="revoke-keys"
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293305    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9c5739-127d-42e6-8190-bbd10a59e43e" containerName="revoke-keys"
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293330    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293337    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293346    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:44.293384 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293354    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:44.293699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293417    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:44.293699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.293431    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="aa9c5739-127d-42e6-8190-bbd10a59e43e" containerName="revoke-keys"
Sep 16 14:37:44.295434 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.295416    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.298229 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.298194    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-363d71-openshift-default-dockercfg-kbbq9\""
Sep 16 14:37:44.308602 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.308580    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423429    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423500    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423532    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423589    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423644    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423671    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423707    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423764    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.423974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.423828    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tbcjn\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525367 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525333    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525413    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525446    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525466    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525622    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525708    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525786    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tbcjn\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.525826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525791    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.526026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525864    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.526026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525889    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.526026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525921    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.526026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.525953    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.526482 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.526464    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.527965 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.527944    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.528120 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.528098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.534895 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.534870    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.535054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.535035    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tbcjn\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn\") pod \"e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") " pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.607427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.607341    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:44.886958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.886929    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:37:44.888801 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:37:44.888775    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbef0cc4c_997f_430c_b9d1_2c3ac0576563.slice/crio-83ee072868e5a0fa512302b0134be3faff28cf71fe42e110e0f3094e3524ca7d WatchSource:0}: Error finding container 83ee072868e5a0fa512302b0134be3faff28cf71fe42e110e0f3094e3524ca7d: Status 404 returned error can't find the container with id 83ee072868e5a0fa512302b0134be3faff28cf71fe42e110e0f3094e3524ca7d
Sep 16 14:37:44.891277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.891233    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:44.891349 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.891326    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:44.891399 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:44.891368    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:45.068632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:45.068591    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerStarted","Data":"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"}
Sep 16 14:37:45.068800 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:45.068660    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerStarted","Data":"83ee072868e5a0fa512302b0134be3faff28cf71fe42e110e0f3094e3524ca7d"}
Sep 16 14:37:45.091523 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:45.091475    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podStartSLOduration=1.091459426 podStartE2EDuration="1.091459426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:37:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:45.088866762 +0000 UTC m=+1786.495447008" watchObservedRunningTime="2026-09-16 14:37:45.091459426 +0000 UTC m=+1786.498039634"
Sep 16 14:37:45.607961 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:45.607924    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:45.613287 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:45.613257    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:46.074494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:46.074459    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:46.075589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:46.075542    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:37:50.931319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:50.931283    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:37:50.931779 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:50.931765    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6443ca3-08cd-44d5-b0d2-97eb522cc085" containerName="istio-proxy"
Sep 16 14:37:50.933860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:50.933840    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:50.937691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:50.937675    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-363d71\""
Sep 16 14:37:50.956368 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:50.956344    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:37:51.094003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.093962    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.094328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.094304    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ggv77\" (UniqueName: \"kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.195409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.195375    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.195600 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.195435    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ggv77\" (UniqueName: \"kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.196082 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.196062    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.204132 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.204110    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ggv77\" (UniqueName: \"kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77\") pod \"payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.244061 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.244032    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:51.522592 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:51.522549    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:37:51.525063 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:37:51.525038    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccc9df55_2381_44f7_9f2a_85f5ecb7d1ac.slice/crio-bca44a31b4a2d784062d95a6f4ba3a1de8af3c56c200c42a0b6e16ab7602b1c7 WatchSource:0}: Error finding container bca44a31b4a2d784062d95a6f4ba3a1de8af3c56c200c42a0b6e16ab7602b1c7: Status 404 returned error can't find the container with id bca44a31b4a2d784062d95a6f4ba3a1de8af3c56c200c42a0b6e16ab7602b1c7
Sep 16 14:37:52.109725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:52.109689    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" event={"ID":"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac","Type":"ContainerStarted","Data":"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"}
Sep 16 14:37:52.110105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:52.109744    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" event={"ID":"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac","Type":"ContainerStarted","Data":"bca44a31b4a2d784062d95a6f4ba3a1de8af3c56c200c42a0b6e16ab7602b1c7"}
Sep 16 14:37:52.110105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:52.109776    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:37:52.131878 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:52.131828    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" podStartSLOduration=2.131813136 podStartE2EDuration="2.131813136s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:37:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:52.130871642 +0000 UTC m=+1793.537451861" watchObservedRunningTime="2026-09-16 14:37:52.131813136 +0000 UTC m=+1793.538393381"
Sep 16 14:37:56.903278 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:56.903242    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:37:56.906540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:56.906510    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:37:56.906815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:56.906638    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:56.909889 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:56.909865    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-a2fada-openshift-default-dockercfg-54jch\""
Sep 16 14:37:57.058681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058630    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058681 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058688    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058718    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmlfh\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058760    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058841    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058884    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.058956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058914    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.059176 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.058970    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.059176 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.059032    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.159859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.159771    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.159859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.159825    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.159863    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.159920    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.159981    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160015    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160040    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160089    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmlfh\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160246    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160313    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160352    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160357    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160674 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160603    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.160736 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.160701    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.162190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.162169    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.162522 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.162502    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.169295 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.169270    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmlfh\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.175578 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.172980    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token\") pod \"e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") " pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.217054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.217018    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:57.522355 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.522328    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:37:57.526018 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:37:57.525835    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1845b1ca_1931_4672_8435_1fa4e5354d8e.slice/crio-b51c090981cb25c1840f57a3591427ca56dc93881c2e4eb95a7b049ae8007c8a WatchSource:0}: Error finding container b51c090981cb25c1840f57a3591427ca56dc93881c2e4eb95a7b049ae8007c8a: Status 404 returned error can't find the container with id b51c090981cb25c1840f57a3591427ca56dc93881c2e4eb95a7b049ae8007c8a
Sep 16 14:37:57.528973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.528931    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:57.529052 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.529016    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:57.529098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:57.529060    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:37:58.145088 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.145053    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerStarted","Data":"effe4ff64ed08fef911e6ccf53b89d8974f154034fc14dd40c84f072f6f192d0"}
Sep 16 14:37:58.145478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.145108    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerStarted","Data":"b51c090981cb25c1840f57a3591427ca56dc93881c2e4eb95a7b049ae8007c8a"}
Sep 16 14:37:58.167853 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.167799    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podStartSLOduration=2.167779474 podStartE2EDuration="2.167779474s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:37:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:37:58.166441418 +0000 UTC m=+1799.573021627" watchObservedRunningTime="2026-09-16 14:37:58.167779474 +0000 UTC m=+1799.574359682"
Sep 16 14:37:58.217216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.217148    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:58.218759 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.218724    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:37:58.218876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:58.218780    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:37:59.222414 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:59.222386    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:37:59.312589 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:59.312543    2580 scope.go:117] "RemoveContainer" containerID="e3d36d584b1606b67d6c982fd6a9a442f294ab853912eb826da2a2e631c796df"
Sep 16 14:37:59.320418 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:37:59.320401    2580 scope.go:117] "RemoveContainer" containerID="5828acdedcf6fb96477ba7ea461407bf7dd49b6bea144dbc7e5d1a53cb12ceba"
Sep 16 14:38:00.155414 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:00.155381    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:00.156272 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:00.156248    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:03.119588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:03.119538    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:38:06.195448 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.195416    2580 generic.go:356] "Generic (PLEG): container finished" podID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerID="c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8" exitCode=0
Sep 16 14:38:06.195905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.195463    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerDied","Data":"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"}
Sep 16 14:38:06.195953 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.195923    2580 scope.go:117] "RemoveContainer" containerID="c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"
Sep 16 14:38:06.197069 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.197045    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:06.197125 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.197113    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:06.197170 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:06.197154    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:07.183059 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.183010    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:38:07.202584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.202532    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerStarted","Data":"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"}
Sep 16 14:38:07.608187 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.608141    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:38:07.608457 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.608435    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 14:38:07.608508 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.608485    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 14:38:07.914697 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.914598    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:38:07.917358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.917334    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:07.919861 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.919836    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-5f0bee-openshift-default-dockercfg-tbdjg\""
Sep 16 14:38:07.935239 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:07.935211    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:38:08.089186 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.089127    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8x4kb\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.089489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.089466    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.089654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.089636    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.089855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.089835    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.089987 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.089973    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.090114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.090100    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.090229 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.090215    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.090331 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.090316    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.090435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.090421    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.191910 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.191870    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192123 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.191930    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192123 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.191969    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192123 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192004    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192171    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192217    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192285    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8x4kb\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192346    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192474 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192383    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.192863 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.192840    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.193108 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.193069    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.193108 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.193086    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.193310 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.193229    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.193310 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.193256    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.195289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.195266    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.195391 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.195374    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.204192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.204165    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.206235 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.206214    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8x4kb\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb\") pod \"e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") " pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.229307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.229270    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:08.556185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.556159    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:38:08.558223 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:38:08.558196    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eacbbd1_2f77_4095_aac9_6387699aa6f1.slice/crio-bcd46166dd19f7ff9851614344cebed767f923132eb5d789408f3f7c2dac6b68 WatchSource:0}: Error finding container bcd46166dd19f7ff9851614344cebed767f923132eb5d789408f3f7c2dac6b68: Status 404 returned error can't find the container with id bcd46166dd19f7ff9851614344cebed767f923132eb5d789408f3f7c2dac6b68
Sep 16 14:38:08.560619 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.560585    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:08.560712 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.560657    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:08.560712 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.560688    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:08.608489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.608464    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 14:38:08.608627 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:08.608517    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 14:38:09.217762 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.217719    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerStarted","Data":"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"}
Sep 16 14:38:09.218230 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.217789    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerStarted","Data":"bcd46166dd19f7ff9851614344cebed767f923132eb5d789408f3f7c2dac6b68"}
Sep 16 14:38:09.230145 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.230112    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:09.235453 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.235423    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:09.259112 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.259063    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podStartSLOduration=2.259043144 podStartE2EDuration="2.259043144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:09.257174991 +0000 UTC m=+1810.663755195" watchObservedRunningTime="2026-09-16 14:38:09.259043144 +0000 UTC m=+1810.665623352"
Sep 16 14:38:09.608409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.608319    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 14:38:09.608409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:09.608373    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 14:38:10.223003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:10.222972    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:10.224003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:10.223985    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:10.608252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:10.608164    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 14:38:10.608252 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:10.608219    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 14:38:11.607703 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:11.607665    2580 patch_prober.go:28] interesting pod/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused" start-of-body=
Sep 16 14:38:11.608098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:11.607718    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.68:15021/healthz/ready\": dial tcp 10.133.0.68:15021: connect: connection refused"
Sep 16 14:38:12.700465 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:12.700416    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:38:12.700989 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:12.700901    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:38:12.701754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:12.701728    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:38:23.306531 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.306494    2580 generic.go:356] "Generic (PLEG): container finished" podID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerID="effe4ff64ed08fef911e6ccf53b89d8974f154034fc14dd40c84f072f6f192d0" exitCode=0
Sep 16 14:38:23.307047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.306576    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerDied","Data":"effe4ff64ed08fef911e6ccf53b89d8974f154034fc14dd40c84f072f6f192d0"}
Sep 16 14:38:23.307047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.307006    2580 scope.go:117] "RemoveContainer" containerID="effe4ff64ed08fef911e6ccf53b89d8974f154034fc14dd40c84f072f6f192d0"
Sep 16 14:38:23.308202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.308176    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:23.308260 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.308240    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:23.308301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.308273    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:23.361809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.361785    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:38:23.364333 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.364314    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.371050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.370828    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-576150\""
Sep 16 14:38:23.383769 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.383736    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:38:23.406644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.406608    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:38:23.410338 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.410313    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.416189 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.416106    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-576150-dockercfg-c5zkz\""
Sep 16 14:38:23.419007 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.418965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:38:23.460152 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.460122    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cr55j\" (UniqueName: \"kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.460350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.460162    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.460350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.460193    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.460459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.460354    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vmgjc\" (UniqueName: \"kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.561753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.561663    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cr55j\" (UniqueName: \"kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.561753 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.561726    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.561991 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.561772    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.561991 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.561848    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vmgjc\" (UniqueName: \"kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.562833 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.562806    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.562903 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.562810    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.582581 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.579926    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cr55j\" (UniqueName: \"kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j\") pod \"payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.593579 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.589667    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vmgjc\" (UniqueName: \"kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc\") pod \"payload-processing-e2e-api-b-576150-69bd9775bf-txq6k\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") " pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:23.722911 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.722879    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:23.732340 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:23.732320    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:24.034863 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.034830    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:38:24.043396 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.043372    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:38:24.047718 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:38:24.047682    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0eaee932_9d89_431c_839d_c7b1dc6b52fa.slice/crio-4764168cb1d923132cdf79c94dd4d7482deb99c91d2479438dd4bd79b12ca211 WatchSource:0}: Error finding container 4764168cb1d923132cdf79c94dd4d7482deb99c91d2479438dd4bd79b12ca211: Status 404 returned error can't find the container with id 4764168cb1d923132cdf79c94dd4d7482deb99c91d2479438dd4bd79b12ca211
Sep 16 14:38:24.314913 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.314874    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" event={"ID":"1a27cf78-9391-4b58-bfc2-19e7f7f012ca","Type":"ContainerStarted","Data":"001c3a0fe763d24eecac80834e89091be700524afac4b3208f98d9fd5642dbb3"}
Sep 16 14:38:24.315656 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.315608    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" event={"ID":"1a27cf78-9391-4b58-bfc2-19e7f7f012ca","Type":"ContainerStarted","Data":"9e9309fbea021e155e08f39b4d9ff9f33b685be75e4a4ba8c96ed921d460a1cd"}
Sep 16 14:38:24.315792 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.315660    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:24.316832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.316765    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" event={"ID":"0eaee932-9d89-431c-839d-c7b1dc6b52fa","Type":"ContainerStarted","Data":"7d8a917c3596471cb8586d5e14d73945fd569bc8d45b7b64b48c5584362a43f5"}
Sep 16 14:38:24.316832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.316803    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" event={"ID":"0eaee932-9d89-431c-839d-c7b1dc6b52fa","Type":"ContainerStarted","Data":"4764168cb1d923132cdf79c94dd4d7482deb99c91d2479438dd4bd79b12ca211"}
Sep 16 14:38:24.317005 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.316853    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:24.319128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.319109    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerStarted","Data":"90b4a053aa80e51b80c0ee81a07927225223e3d1f20357b1c2eaf8c146825db1"}
Sep 16 14:38:24.332454 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.332408    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" podStartSLOduration=1.332392121 podStartE2EDuration="1.332392121s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:24.332131207 +0000 UTC m=+1825.738711429" watchObservedRunningTime="2026-09-16 14:38:24.332392121 +0000 UTC m=+1825.738972329"
Sep 16 14:38:24.369648 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:24.369591    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" podStartSLOduration=1.369551898 podStartE2EDuration="1.369551898s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:38:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:38:24.363742612 +0000 UTC m=+1825.770322819" watchObservedRunningTime="2026-09-16 14:38:24.369551898 +0000 UTC m=+1825.776132110"
Sep 16 14:38:25.217654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:25.217620    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:25.217879 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:25.217858    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:25.217929 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:25.217905    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:26.218195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:26.218156    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:26.219530 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:26.219491    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:27.218283 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:27.218245    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:27.218860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:27.218826    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:28.218336 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:28.218286    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:28.218892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:28.218376    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:29.217905 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:29.217868    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:29.218080 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:29.217932    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:30.156071 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:30.156027    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:30.217732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:30.217695    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused" start-of-body=
Sep 16 14:38:30.217936 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:30.217759    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.70:15021/healthz/ready\": dial tcp 10.133.0.70:15021: connect: connection refused"
Sep 16 14:38:31.221606 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:31.221541    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:31.222716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:31.222699    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:38:32.365549 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.365519    2580 generic.go:356] "Generic (PLEG): container finished" podID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerID="e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4" exitCode=0
Sep 16 14:38:32.366006 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.365594    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerDied","Data":"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"}
Sep 16 14:38:32.366070 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.366055    2580 scope.go:117] "RemoveContainer" containerID="e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"
Sep 16 14:38:32.367275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.367246    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:32.367376 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.367333    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:32.367439 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:32.367382    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:38:33.376321 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:33.376284    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerStarted","Data":"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"}
Sep 16 14:38:34.230492 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:34.230450    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:34.230722 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:34.230693    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:38:34.230773 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:34.230741    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 16 14:38:35.230705 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:35.230668    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:38:35.231102 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:35.230723    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 16 14:38:35.326826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:35.326798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:38:36.230522 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:36.230488    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:38:36.230734 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:36.230541    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 16 14:38:36.333341 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:36.333311    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:38:36.776632 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:36.776601    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:38:36.957935 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:36.957901    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:38:37.229938 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:37.229902    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:38:37.230120 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:37.229972    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 16 14:38:38.230543 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:38.230504    2580 patch_prober.go:28] interesting pod/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused" start-of-body=
Sep 16 14:38:38.231045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:38.230602    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.71:15021/healthz/ready\": dial tcp 10.133.0.71:15021: connect: connection refused"
Sep 16 14:38:39.234505 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:39.234479    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:39.234930 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:39.234798    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:38:39.235431 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:38:39.235413    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:39:14.494232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:14.494191    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:39:14.494811 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:14.494437    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" containerID="cri-o://90b4a053aa80e51b80c0ee81a07927225223e3d1f20357b1c2eaf8c146825db1" gracePeriod=30
Sep 16 14:39:16.224552 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:16.224513    2580 patch_prober.go:28] interesting pod/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:39:16.224946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:16.224594    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:39:19.672822 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.672788    2580 generic.go:356] "Generic (PLEG): container finished" podID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerID="90b4a053aa80e51b80c0ee81a07927225223e3d1f20357b1c2eaf8c146825db1" exitCode=0
Sep 16 14:39:19.673262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.672836    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerDied","Data":"90b4a053aa80e51b80c0ee81a07927225223e3d1f20357b1c2eaf8c146825db1"}
Sep 16 14:39:19.673262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.672896    2580 scope.go:117] "RemoveContainer" containerID="effe4ff64ed08fef911e6ccf53b89d8974f154034fc14dd40c84f072f6f192d0"
Sep 16 14:39:19.737061 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.737039    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:39:19.781678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781613    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781650    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781695    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781717    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781745    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781762    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781806    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.781862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781845    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.782190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.781907    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmlfh\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh\") pod \"1845b1ca-1931-4672-8435-1fa4e5354d8e\" (UID: \"1845b1ca-1931-4672-8435-1fa4e5354d8e\") "
Sep 16 14:39:19.782190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782007    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data" (OuterVolumeSpecName: "istio-data") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:39:19.782190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782035    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:39:19.782359 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782257    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:39:19.782419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782351    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:19.782419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782360    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.782419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782381    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.782419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782394    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.782419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.782388    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:39:19.783892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.783863    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:39:19.784086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.784062    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh" (OuterVolumeSpecName: "kube-api-access-xmlfh") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "kube-api-access-xmlfh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:19.784186 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.784170    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:39:19.784353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.784338    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token" (OuterVolumeSpecName: "istio-token") pod "1845b1ca-1931-4672-8435-1fa4e5354d8e" (UID: "1845b1ca-1931-4672-8435-1fa4e5354d8e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:19.883192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883163    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.883192 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883193    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1845b1ca-1931-4672-8435-1fa4e5354d8e-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.883420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883204    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.883420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883215    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmlfh\" (UniqueName: \"kubernetes.io/projected/1845b1ca-1931-4672-8435-1fa4e5354d8e-kube-api-access-xmlfh\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.883420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883227    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:19.883420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:19.883238    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1845b1ca-1931-4672-8435-1fa4e5354d8e-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:20.679638 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:20.679600    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"
Sep 16 14:39:20.679638 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:20.679627    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9" event={"ID":"1845b1ca-1931-4672-8435-1fa4e5354d8e","Type":"ContainerDied","Data":"b51c090981cb25c1840f57a3591427ca56dc93881c2e4eb95a7b049ae8007c8a"}
Sep 16 14:39:20.680105 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:20.679708    2580 scope.go:117] "RemoveContainer" containerID="90b4a053aa80e51b80c0ee81a07927225223e3d1f20357b1c2eaf8c146825db1"
Sep 16 14:39:20.706245 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:20.706216    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:39:20.708227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:20.708205    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-a2fada-openshift-default-bf5bb8d98-rttb9"]
Sep 16 14:39:21.185520 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:21.185479    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" path="/var/lib/kubelet/pods/1845b1ca-1931-4672-8435-1fa4e5354d8e/volumes"
Sep 16 14:39:24.178227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:24.178198    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:39:35.483539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.483510    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:39:35.512926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.512538    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:39:35.512926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.512816    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" podUID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" containerName="payload-processing" containerID="cri-o://7d8a917c3596471cb8586d5e14d73945fd569bc8d45b7b64b48c5584362a43f5" gracePeriod=30
Sep 16 14:39:35.516470 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.516435    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:39:35.516722 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.516681    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" podUID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" containerName="payload-pre-processing" containerID="cri-o://001c3a0fe763d24eecac80834e89091be700524afac4b3208f98d9fd5642dbb3" gracePeriod=30
Sep 16 14:39:35.604248 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.604225    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:39:35.772202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.772093    2580 generic.go:356] "Generic (PLEG): container finished" podID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" containerID="7d8a917c3596471cb8586d5e14d73945fd569bc8d45b7b64b48c5584362a43f5" exitCode=0
Sep 16 14:39:35.772202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.772178    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" event={"ID":"0eaee932-9d89-431c-839d-c7b1dc6b52fa","Type":"ContainerDied","Data":"7d8a917c3596471cb8586d5e14d73945fd569bc8d45b7b64b48c5584362a43f5"}
Sep 16 14:39:35.774086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.774060    2580 generic.go:356] "Generic (PLEG): container finished" podID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" containerID="001c3a0fe763d24eecac80834e89091be700524afac4b3208f98d9fd5642dbb3" exitCode=0
Sep 16 14:39:35.774257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.774093    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" event={"ID":"1a27cf78-9391-4b58-bfc2-19e7f7f012ca","Type":"ContainerDied","Data":"001c3a0fe763d24eecac80834e89091be700524afac4b3208f98d9fd5642dbb3"}
Sep 16 14:39:35.801274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.801256    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:39:35.804664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.804646    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:39:35.948825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.948795    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vmgjc\" (UniqueName: \"kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc\") pod \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") "
Sep 16 14:39:35.949040 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.948926    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume\") pod \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") "
Sep 16 14:39:35.949040 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.948959    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cr55j\" (UniqueName: \"kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j\") pod \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\" (UID: \"1a27cf78-9391-4b58-bfc2-19e7f7f012ca\") "
Sep 16 14:39:35.949040 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.949000    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume\") pod \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\" (UID: \"0eaee932-9d89-431c-839d-c7b1dc6b52fa\") "
Sep 16 14:39:35.949344 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.949315    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a27cf78-9391-4b58-bfc2-19e7f7f012ca" (UID: "1a27cf78-9391-4b58-bfc2-19e7f7f012ca"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:35.949422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.949353    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "0eaee932-9d89-431c-839d-c7b1dc6b52fa" (UID: "0eaee932-9d89-431c-839d-c7b1dc6b52fa"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:35.949422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.949410    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:35.950902 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.950880    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc" (OuterVolumeSpecName: "kube-api-access-vmgjc") pod "0eaee932-9d89-431c-839d-c7b1dc6b52fa" (UID: "0eaee932-9d89-431c-839d-c7b1dc6b52fa"). InnerVolumeSpecName "kube-api-access-vmgjc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:35.951127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:35.951108    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j" (OuterVolumeSpecName: "kube-api-access-cr55j") pod "1a27cf78-9391-4b58-bfc2-19e7f7f012ca" (UID: "1a27cf78-9391-4b58-bfc2-19e7f7f012ca"). InnerVolumeSpecName "kube-api-access-cr55j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:36.050537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.050471    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/0eaee932-9d89-431c-839d-c7b1dc6b52fa-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:36.050537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.050495    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vmgjc\" (UniqueName: \"kubernetes.io/projected/0eaee932-9d89-431c-839d-c7b1dc6b52fa-kube-api-access-vmgjc\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:36.050537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.050504    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cr55j\" (UniqueName: \"kubernetes.io/projected/1a27cf78-9391-4b58-bfc2-19e7f7f012ca-kube-api-access-cr55j\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:36.435362 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435276    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:39:36.435865 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435846    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:36.435865 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435866    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435884    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" containerName="payload-processing"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435890    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" containerName="payload-processing"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435898    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435904    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435914    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" containerName="payload-pre-processing"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435919    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" containerName="payload-pre-processing"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435982    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435992    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" containerName="payload-pre-processing"
Sep 16 14:39:36.436004 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.435999    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" containerName="payload-processing"
Sep 16 14:39:36.438025 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.438006    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.440318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.440288    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-cleanup-d5498f-openshift-default-dockercfg-q7jng\""
Sep 16 14:39:36.494545 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.494517    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:39:36.555528 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.555438    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.555822 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.555797    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556026 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556008    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556194 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556179    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg4x9\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556362 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556346    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556517 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556503    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556678    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.556875 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.556861    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.557060 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.557045    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658006 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.657971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658020    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg4x9\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658087    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658119    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658148    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658214    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658295    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658382    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658410    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658641 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658618    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658737 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658713    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658809 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658748    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658869 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658804    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.658971 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.658952    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.660452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.660430    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.660595 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.660578    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.666445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.666422    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.666570 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.666543    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg4x9\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9\") pod \"e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") " pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.749625 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.749602    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:36.783008 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.782984    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"
Sep 16 14:39:36.783163 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.783021    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k" event={"ID":"0eaee932-9d89-431c-839d-c7b1dc6b52fa","Type":"ContainerDied","Data":"4764168cb1d923132cdf79c94dd4d7482deb99c91d2479438dd4bd79b12ca211"}
Sep 16 14:39:36.783163 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.783078    2580 scope.go:117] "RemoveContainer" containerID="7d8a917c3596471cb8586d5e14d73945fd569bc8d45b7b64b48c5584362a43f5"
Sep 16 14:39:36.784646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.784615    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx" event={"ID":"1a27cf78-9391-4b58-bfc2-19e7f7f012ca","Type":"ContainerDied","Data":"9e9309fbea021e155e08f39b4d9ff9f33b685be75e4a4ba8c96ed921d460a1cd"}
Sep 16 14:39:36.784646 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.784639    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"
Sep 16 14:39:36.793502 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.793140    2580 scope.go:117] "RemoveContainer" containerID="001c3a0fe763d24eecac80834e89091be700524afac4b3208f98d9fd5642dbb3"
Sep 16 14:39:36.819608 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.814447    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:39:36.829098 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.828951    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-576150-6788878894-m2kmx"]
Sep 16 14:39:36.848318 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.848186    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:39:36.850328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:36.850299    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-576150-69bd9775bf-txq6k"]
Sep 16 14:39:37.042083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.042060    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:39:37.044125 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:39:37.044098    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podccb21e37_f739_41e8_8305_65f13cd6896b.slice/crio-a81af0531f3b5c435a3317e44a992c6d4a72e6b304d1b8d6ac303af0193b7904 WatchSource:0}: Error finding container a81af0531f3b5c435a3317e44a992c6d4a72e6b304d1b8d6ac303af0193b7904: Status 404 returned error can't find the container with id a81af0531f3b5c435a3317e44a992c6d4a72e6b304d1b8d6ac303af0193b7904
Sep 16 14:39:37.046301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.046270    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:39:37.046372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.046336    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:39:37.046372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.046363    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:39:37.185573 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.185534    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0eaee932-9d89-431c-839d-c7b1dc6b52fa" path="/var/lib/kubelet/pods/0eaee932-9d89-431c-839d-c7b1dc6b52fa/volumes"
Sep 16 14:39:37.185940 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.185925    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a27cf78-9391-4b58-bfc2-19e7f7f012ca" path="/var/lib/kubelet/pods/1a27cf78-9391-4b58-bfc2-19e7f7f012ca/volumes"
Sep 16 14:39:37.793637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.793602    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerStarted","Data":"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"}
Sep 16 14:39:37.794017 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.793650    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerStarted","Data":"a81af0531f3b5c435a3317e44a992c6d4a72e6b304d1b8d6ac303af0193b7904"}
Sep 16 14:39:37.818319 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:37.818269    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podStartSLOduration=1.818253049 podStartE2EDuration="1.818253049s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:39:36 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:39:37.815995505 +0000 UTC m=+1899.222575712" watchObservedRunningTime="2026-09-16 14:39:37.818253049 +0000 UTC m=+1899.224833256"
Sep 16 14:39:38.750320 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:38.750290    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:38.755289 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:38.755266    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:38.799194 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:38.799163    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:38.800114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:38.800094    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:39:42.607806 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.607774    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:39:42.608310 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.608296    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1845b1ca-1931-4672-8435-1fa4e5354d8e" containerName="istio-proxy"
Sep 16 14:39:42.610767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.610753    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.613929 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.613905    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-cleanup-d5498f-dockercfg-htxtl\""
Sep 16 14:39:42.614050 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.614022    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d5498f\""
Sep 16 14:39:42.621664 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.621642    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:39:42.718452 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.718407    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpm68\" (UniqueName: \"kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.718666 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.718518    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.819334 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.819297    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mpm68\" (UniqueName: \"kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.819504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.819397    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.820117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.820098    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.828542 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.828519    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpm68\" (UniqueName: \"kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68\") pod \"payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:42.922710 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:42.922643    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:43.222280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:43.222255    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:39:43.224092 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:39:43.224061    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod889b9dd6_3da4_4727_a394_5afef24d3fc5.slice/crio-1b6159aae217c5ac6c4186e5a7812e0c89c3f0028cc1992cad2e82c5ef83d959 WatchSource:0}: Error finding container 1b6159aae217c5ac6c4186e5a7812e0c89c3f0028cc1992cad2e82c5ef83d959: Status 404 returned error can't find the container with id 1b6159aae217c5ac6c4186e5a7812e0c89c3f0028cc1992cad2e82c5ef83d959
Sep 16 14:39:43.834251 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:43.834220    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" event={"ID":"889b9dd6-3da4-4727-a394-5afef24d3fc5","Type":"ContainerStarted","Data":"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"}
Sep 16 14:39:43.834665 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:43.834269    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" event={"ID":"889b9dd6-3da4-4727-a394-5afef24d3fc5","Type":"ContainerStarted","Data":"1b6159aae217c5ac6c4186e5a7812e0c89c3f0028cc1992cad2e82c5ef83d959"}
Sep 16 14:39:43.834665 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:43.834324    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:43.863109 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:43.863069    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" podStartSLOduration=1.863054247 podStartE2EDuration="1.863054247s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:39:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:39:43.861720275 +0000 UTC m=+1905.268300520" watchObservedRunningTime="2026-09-16 14:39:43.863054247 +0000 UTC m=+1905.269634455"
Sep 16 14:39:54.842345 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:54.842315    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:39:55.814195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:55.814162    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:39:55.814442 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:55.814411    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" podUID="c69b163d-6eea-4a02-b9c0-5c995d907840" containerName="payload-processing" containerID="cri-o://3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836" gracePeriod=30
Sep 16 14:39:55.823045 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:55.823019    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:39:55.823203 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:55.823184    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" podUID="906246a1-0c08-4a64-aad3-bea251617860" containerName="payload-pre-processing" containerID="cri-o://103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77" gracePeriod=30
Sep 16 14:39:56.209338 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.209039    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:39:56.229362 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.229338    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:39:56.249541 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.249507    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume\") pod \"c69b163d-6eea-4a02-b9c0-5c995d907840\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") "
Sep 16 14:39:56.249716 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.249645    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b5pxj\" (UniqueName: \"kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj\") pod \"c69b163d-6eea-4a02-b9c0-5c995d907840\" (UID: \"c69b163d-6eea-4a02-b9c0-5c995d907840\") "
Sep 16 14:39:56.249912 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.249889    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "c69b163d-6eea-4a02-b9c0-5c995d907840" (UID: "c69b163d-6eea-4a02-b9c0-5c995d907840"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:56.251703 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.251675    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj" (OuterVolumeSpecName: "kube-api-access-b5pxj") pod "c69b163d-6eea-4a02-b9c0-5c995d907840" (UID: "c69b163d-6eea-4a02-b9c0-5c995d907840"). InnerVolumeSpecName "kube-api-access-b5pxj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:56.350441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.350394    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tsm74\" (UniqueName: \"kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74\") pod \"906246a1-0c08-4a64-aad3-bea251617860\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") "
Sep 16 14:39:56.350650 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.350547    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume\") pod \"906246a1-0c08-4a64-aad3-bea251617860\" (UID: \"906246a1-0c08-4a64-aad3-bea251617860\") "
Sep 16 14:39:56.350775 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.350763    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/c69b163d-6eea-4a02-b9c0-5c995d907840-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:56.350815 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.350777    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b5pxj\" (UniqueName: \"kubernetes.io/projected/c69b163d-6eea-4a02-b9c0-5c995d907840-kube-api-access-b5pxj\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:56.350887 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.350864    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "906246a1-0c08-4a64-aad3-bea251617860" (UID: "906246a1-0c08-4a64-aad3-bea251617860"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:39:56.352504 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.352484    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74" (OuterVolumeSpecName: "kube-api-access-tsm74") pod "906246a1-0c08-4a64-aad3-bea251617860" (UID: "906246a1-0c08-4a64-aad3-bea251617860"). InnerVolumeSpecName "kube-api-access-tsm74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:39:56.452055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.452024    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/906246a1-0c08-4a64-aad3-bea251617860-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:56.452055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.452054    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tsm74\" (UniqueName: \"kubernetes.io/projected/906246a1-0c08-4a64-aad3-bea251617860-kube-api-access-tsm74\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:39:56.915424 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.915391    2580 generic.go:356] "Generic (PLEG): container finished" podID="c69b163d-6eea-4a02-b9c0-5c995d907840" containerID="3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836" exitCode=0
Sep 16 14:39:56.915636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.915436    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"
Sep 16 14:39:56.915636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.915474    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" event={"ID":"c69b163d-6eea-4a02-b9c0-5c995d907840","Type":"ContainerDied","Data":"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"}
Sep 16 14:39:56.915636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.915549    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2" event={"ID":"c69b163d-6eea-4a02-b9c0-5c995d907840","Type":"ContainerDied","Data":"707631d52920b66b99b880fe0a14a50619ade7a25f4912edfa98105eb6bbbf06"}
Sep 16 14:39:56.915636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.915597    2580 scope.go:117] "RemoveContainer" containerID="3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"
Sep 16 14:39:56.917003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.916917    2580 generic.go:356] "Generic (PLEG): container finished" podID="906246a1-0c08-4a64-aad3-bea251617860" containerID="103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77" exitCode=0
Sep 16 14:39:56.917003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.916941    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" event={"ID":"906246a1-0c08-4a64-aad3-bea251617860","Type":"ContainerDied","Data":"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"}
Sep 16 14:39:56.917003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.916958    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"
Sep 16 14:39:56.917003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.916991    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z" event={"ID":"906246a1-0c08-4a64-aad3-bea251617860","Type":"ContainerDied","Data":"4ca6c86cfa7f7e8a23daf62220323cb59e616bae3f5b9f3fe28d9a46adbaea17"}
Sep 16 14:39:56.926629 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.926609    2580 scope.go:117] "RemoveContainer" containerID="3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"
Sep 16 14:39:56.926981 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:39:56.926906    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836\": container with ID starting with 3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836 not found: ID does not exist" containerID="3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"
Sep 16 14:39:56.926981 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.926936    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836"} err="failed to get container status \"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836\": rpc error: code = NotFound desc = could not find container \"3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836\": container with ID starting with 3fc79d0fb57330f67a813e20c1cf448f2748ef2a0a86ea839b0852a8e407e836 not found: ID does not exist"
Sep 16 14:39:56.926981 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.926953    2580 scope.go:117] "RemoveContainer" containerID="103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"
Sep 16 14:39:56.934872 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.934850    2580 scope.go:117] "RemoveContainer" containerID="103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"
Sep 16 14:39:56.935133 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:39:56.935115    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77\": container with ID starting with 103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77 not found: ID does not exist" containerID="103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"
Sep 16 14:39:56.935181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.935140    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77"} err="failed to get container status \"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77\": rpc error: code = NotFound desc = could not find container \"103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77\": container with ID starting with 103d2c83a9e3aa25b08350b7e4f0d1021a9eed42aa63ce05069e415f4becce77 not found: ID does not exist"
Sep 16 14:39:56.941154 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.940640    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:39:56.943266 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.943243    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-dc3e5b-7b89b948c5-4p2t2"]
Sep 16 14:39:56.953537 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.953513    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:39:56.959692 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:56.959671    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-dc3e5b-7948b456cd-khc6z"]
Sep 16 14:39:57.188714 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:57.188619    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="906246a1-0c08-4a64-aad3-bea251617860" path="/var/lib/kubelet/pods/906246a1-0c08-4a64-aad3-bea251617860/volumes"
Sep 16 14:39:57.189209 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:39:57.189185    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c69b163d-6eea-4a02-b9c0-5c995d907840" path="/var/lib/kubelet/pods/c69b163d-6eea-4a02-b9c0-5c995d907840/volumes"
Sep 16 14:40:02.954027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.953993    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerID="f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977" exitCode=0
Sep 16 14:40:02.954440 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.954063    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerDied","Data":"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"}
Sep 16 14:40:02.954489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.954463    2580 scope.go:117] "RemoveContainer" containerID="f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"
Sep 16 14:40:02.955624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.955597    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:40:02.955687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.955664    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:40:02.955725 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:02.955700    2580 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236212Ki","pods":"250"}
Sep 16 14:40:03.964073 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:03.964041    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerStarted","Data":"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"}
Sep 16 14:40:04.234570 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.234478    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:40:04.234755 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.234727    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" containerID="cri-o://7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011" gracePeriod=30
Sep 16 14:40:04.750343 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.750307    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:04.751473 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.751447    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:40:04.751617 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.751498    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 16 14:40:04.768416 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.768388    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"]
Sep 16 14:40:04.769262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769239    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="906246a1-0c08-4a64-aad3-bea251617860" containerName="payload-pre-processing"
Sep 16 14:40:04.769405 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769392    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="906246a1-0c08-4a64-aad3-bea251617860" containerName="payload-pre-processing"
Sep 16 14:40:04.769497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769487    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c69b163d-6eea-4a02-b9c0-5c995d907840" containerName="payload-processing"
Sep 16 14:40:04.769608 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769597    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c69b163d-6eea-4a02-b9c0-5c995d907840" containerName="payload-processing"
Sep 16 14:40:04.769840 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769826    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c69b163d-6eea-4a02-b9c0-5c995d907840" containerName="payload-processing"
Sep 16 14:40:04.769935 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.769924    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="906246a1-0c08-4a64-aad3-bea251617860" containerName="payload-pre-processing"
Sep 16 14:40:04.773457 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.773429    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:04.776305 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.776194    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:40:04.776456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.776434    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:40:04.793942 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.793912    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"]
Sep 16 14:40:04.865934 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.865886    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8\" (UID: \"29b49a3a-492c-4617-bd71-cb4af07d025e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:04.966999 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.966959    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8\" (UID: \"29b49a3a-492c-4617-bd71-cb4af07d025e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:04.967890 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:04.967858    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8\" (UID: \"29b49a3a-492c-4617-bd71-cb4af07d025e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:05.094600 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.094487    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:05.366353 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.366325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"]
Sep 16 14:40:05.372118 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:40:05.372062    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b49a3a_492c_4617_bd71_cb4af07d025e.slice/crio-f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670 WatchSource:0}: Error finding container f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670: Status 404 returned error can't find the container with id f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670
Sep 16 14:40:05.750227 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.750187    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:40:05.750381 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.750252    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 16 14:40:05.983079 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.982219    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8" event={"ID":"29b49a3a-492c-4617-bd71-cb4af07d025e","Type":"ContainerStarted","Data":"f963f8f401117577a1714235f16540c0636e20c1a26dbdf589879dc05636a8b8"}
Sep 16 14:40:05.983079 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:05.982287    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8" event={"ID":"29b49a3a-492c-4617-bd71-cb4af07d025e","Type":"ContainerStarted","Data":"f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670"}
Sep 16 14:40:06.000435 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:06.000323    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8" podStartSLOduration=2.000304962 podStartE2EDuration="2.000304962s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:40:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:40:05.999094581 +0000 UTC m=+1927.405674789" watchObservedRunningTime="2026-09-16 14:40:06.000304962 +0000 UTC m=+1927.406885169"
Sep 16 14:40:06.750314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:06.750274    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:40:06.750503 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:06.750338    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 16 14:40:06.993702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:06.993651    2580 generic.go:356] "Generic (PLEG): container finished" podID="29b49a3a-492c-4617-bd71-cb4af07d025e" containerID="f963f8f401117577a1714235f16540c0636e20c1a26dbdf589879dc05636a8b8" exitCode=0
Sep 16 14:40:06.994196 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:06.993756    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8" event={"ID":"29b49a3a-492c-4617-bd71-cb4af07d025e","Type":"ContainerDied","Data":"f963f8f401117577a1714235f16540c0636e20c1a26dbdf589879dc05636a8b8"}
Sep 16 14:40:07.750037 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:07.750000    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:40:07.750221 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:07.750058    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 16 14:40:08.157425 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.157400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:08.199161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.198951    2580 patch_prober.go:28] interesting pod/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:40:08.199161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.199024    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:40:08.223202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.223168    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca\") pod \"29b49a3a-492c-4617-bd71-cb4af07d025e\" (UID: \"29b49a3a-492c-4617-bd71-cb4af07d025e\") "
Sep 16 14:40:08.223577 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.223517    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "29b49a3a-492c-4617-bd71-cb4af07d025e" (UID: "29b49a3a-492c-4617-bd71-cb4af07d025e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:08.223719 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.223625    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/29b49a3a-492c-4617-bd71-cb4af07d025e-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:08.750202 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.750170    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused" start-of-body=
Sep 16 14:40:08.750386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.750251    2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.74:15021/healthz/ready\": dial tcp 10.133.0.74:15021: connect: connection refused"
Sep 16 14:40:08.799709 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:08.799671    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:09.007077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.006988    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8" event={"ID":"29b49a3a-492c-4617-bd71-cb4af07d025e","Type":"ContainerDied","Data":"f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670"}
Sep 16 14:40:09.007077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.007048    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4da750f0a2ad94ce561c9e43cac4851174977bb77cf8662928584f5c6663670"
Sep 16 14:40:09.007077 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.006984    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"
Sep 16 14:40:09.483619 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.483594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:40:09.536295 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536267    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536466 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536335    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536466 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536410    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536466 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536450    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536490    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536508    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536528    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536586    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536615    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536643    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data\") pod \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\" (UID: \"d96c0af5-b2f9-44a6-9853-0ccf1ead036f\") "
Sep 16 14:40:09.536966 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536920    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:09.537025 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.536960    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:09.537100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.537072    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.537100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.537095    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.537206 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.537108    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.537206 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.537163    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:09.537206 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.537183    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data" (OuterVolumeSpecName: "istio-data") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:09.538824 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.538800    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token" (OuterVolumeSpecName: "istio-token") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:09.538987 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.538967    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:40:09.539386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.539348    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:09.539637 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.539443    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj" (OuterVolumeSpecName: "kube-api-access-52whj") pod "d96c0af5-b2f9-44a6-9853-0ccf1ead036f" (UID: "d96c0af5-b2f9-44a6-9853-0ccf1ead036f"). InnerVolumeSpecName "kube-api-access-52whj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637603    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637635    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637644    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637652    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637661    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-52whj\" (UniqueName: \"kubernetes.io/projected/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-kube-api-access-52whj\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.637691 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.637671    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d96c0af5-b2f9-44a6-9853-0ccf1ead036f-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:09.753955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.753921    2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:09.755358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:09.755333    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:10.014532 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.014498    2580 generic.go:356] "Generic (PLEG): container finished" podID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerID="7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011" exitCode=0
Sep 16 14:40:10.014732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.014574    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"
Sep 16 14:40:10.014732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.014587    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerDied","Data":"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"}
Sep 16 14:40:10.014732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.014649    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv" event={"ID":"d96c0af5-b2f9-44a6-9853-0ccf1ead036f","Type":"ContainerDied","Data":"0e4cc27700f92c0b1a977a30952dcebcfbf309b78c65959556d3f9c8eff4626a"}
Sep 16 14:40:10.014732 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.014680    2580 scope.go:117] "RemoveContainer" containerID="7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"
Sep 16 14:40:10.024712 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.024697    2580 scope.go:117] "RemoveContainer" containerID="b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"
Sep 16 14:40:10.032366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.032347    2580 scope.go:117] "RemoveContainer" containerID="7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"
Sep 16 14:40:10.032778 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:10.032743    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011\": container with ID starting with 7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011 not found: ID does not exist" containerID="7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"
Sep 16 14:40:10.032870 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.032788    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011"} err="failed to get container status \"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011\": rpc error: code = NotFound desc = could not find container \"7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011\": container with ID starting with 7225bf6d154812d740abc7db0f07aec6e3705f4d72daafc67705d270a33f7011 not found: ID does not exist"
Sep 16 14:40:10.032870 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.032811    2580 scope.go:117] "RemoveContainer" containerID="b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"
Sep 16 14:40:10.033078 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:10.033062    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a\": container with ID starting with b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a not found: ID does not exist" containerID="b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"
Sep 16 14:40:10.033131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.033083    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a"} err="failed to get container status \"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a\": rpc error: code = NotFound desc = could not find container \"b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a\": container with ID starting with b53d1552b0eb2c091b2c61ad46966c5ba4cfbeeec81719a52b4e31caeca92e4a not found: ID does not exist"
Sep 16 14:40:10.039115 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.039096    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:40:10.045114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:10.045074    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-dc3e5b-openshift-default-5ffd9b6d68-5ltcv"]
Sep 16 14:40:11.194495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:11.194453    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" path="/var/lib/kubelet/pods/d96c0af5-b2f9-44a6-9853-0ccf1ead036f/volumes"
Sep 16 14:40:12.287534 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.287497    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:40:12.288020 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.287745    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" podUID="889b9dd6-3da4-4727-a394-5afef24d3fc5" containerName="payload-pre-processing" containerID="cri-o://2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02" gracePeriod=30
Sep 16 14:40:12.581434 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.581400    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:40:12.669394 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.669359    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume\") pod \"889b9dd6-3da4-4727-a394-5afef24d3fc5\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") "
Sep 16 14:40:12.669616 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.669431    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpm68\" (UniqueName: \"kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68\") pod \"889b9dd6-3da4-4727-a394-5afef24d3fc5\" (UID: \"889b9dd6-3da4-4727-a394-5afef24d3fc5\") "
Sep 16 14:40:12.669849 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.669826    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "889b9dd6-3da4-4727-a394-5afef24d3fc5" (UID: "889b9dd6-3da4-4727-a394-5afef24d3fc5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:12.671761 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.671702    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68" (OuterVolumeSpecName: "kube-api-access-mpm68") pod "889b9dd6-3da4-4727-a394-5afef24d3fc5" (UID: "889b9dd6-3da4-4727-a394-5afef24d3fc5"). InnerVolumeSpecName "kube-api-access-mpm68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:12.770715 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.770677    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpm68\" (UniqueName: \"kubernetes.io/projected/889b9dd6-3da4-4727-a394-5afef24d3fc5-kube-api-access-mpm68\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:12.770715 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:12.770705    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/889b9dd6-3da4-4727-a394-5afef24d3fc5-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:13.032326 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.032291    2580 generic.go:356] "Generic (PLEG): container finished" podID="889b9dd6-3da4-4727-a394-5afef24d3fc5" containerID="2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02" exitCode=0
Sep 16 14:40:13.032540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.032338    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"
Sep 16 14:40:13.032540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.032381    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" event={"ID":"889b9dd6-3da4-4727-a394-5afef24d3fc5","Type":"ContainerDied","Data":"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"}
Sep 16 14:40:13.032540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.032442    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd" event={"ID":"889b9dd6-3da4-4727-a394-5afef24d3fc5","Type":"ContainerDied","Data":"1b6159aae217c5ac6c4186e5a7812e0c89c3f0028cc1992cad2e82c5ef83d959"}
Sep 16 14:40:13.032540 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.032473    2580 scope.go:117] "RemoveContainer" containerID="2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"
Sep 16 14:40:13.041627 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.041610    2580 scope.go:117] "RemoveContainer" containerID="2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"
Sep 16 14:40:13.041901 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:13.041881    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02\": container with ID starting with 2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02 not found: ID does not exist" containerID="2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"
Sep 16 14:40:13.041941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.041909    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02"} err="failed to get container status \"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02\": rpc error: code = NotFound desc = could not find container \"2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02\": container with ID starting with 2fbee70f5803e78dd6bdbd9e6f50a3509651b6203e01c0f30e3acf1d783d6e02 not found: ID does not exist"
Sep 16 14:40:13.056695 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.056657    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:40:13.063798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.063769    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-cleanup-d5498f-6657bfd4f8-zvwmd"]
Sep 16 14:40:13.185478 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:13.185446    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="889b9dd6-3da4-4727-a394-5afef24d3fc5" path="/var/lib/kubelet/pods/889b9dd6-3da4-4727-a394-5afef24d3fc5/volumes"
Sep 16 14:40:15.986837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:15.986801    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:40:15.987476 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:15.987058    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" podUID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" containerName="payload-processing" containerID="cri-o://ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c" gracePeriod=30
Sep 16 14:40:15.996812 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:15.996785    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:40:15.997020 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:15.996994    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" podUID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" containerName="payload-pre-processing" containerID="cri-o://48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd" gracePeriod=30
Sep 16 14:40:16.309785 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.309762    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:40:16.312919 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.312900    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:40:16.396218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.396185    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"]
Sep 16 14:40:16.403825 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.403796    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-bdfd4a-cbfb8"]
Sep 16 14:40:16.407801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.407781    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume\") pod \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") "
Sep 16 14:40:16.407862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.407826    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c44p9\" (UniqueName: \"kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9\") pod \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") "
Sep 16 14:40:16.407862 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.407848    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume\") pod \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\" (UID: \"abf20d47-3f33-4a67-afb7-4ccdd7540cbf\") "
Sep 16 14:40:16.407948 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.407934    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k7tbd\" (UniqueName: \"kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd\") pod \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\" (UID: \"702d5aa7-e543-4d31-9304-c60c06a9dfbf\") "
Sep 16 14:40:16.408308 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.408195    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "abf20d47-3f33-4a67-afb7-4ccdd7540cbf" (UID: "abf20d47-3f33-4a67-afb7-4ccdd7540cbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:16.408409 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.408385    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "702d5aa7-e543-4d31-9304-c60c06a9dfbf" (UID: "702d5aa7-e543-4d31-9304-c60c06a9dfbf"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:16.410106 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.410087    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd" (OuterVolumeSpecName: "kube-api-access-k7tbd") pod "702d5aa7-e543-4d31-9304-c60c06a9dfbf" (UID: "702d5aa7-e543-4d31-9304-c60c06a9dfbf"). InnerVolumeSpecName "kube-api-access-k7tbd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:16.410168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.410115    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9" (OuterVolumeSpecName: "kube-api-access-c44p9") pod "abf20d47-3f33-4a67-afb7-4ccdd7540cbf" (UID: "abf20d47-3f33-4a67-afb7-4ccdd7540cbf"). InnerVolumeSpecName "kube-api-access-c44p9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:16.509591 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.509542    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/702d5aa7-e543-4d31-9304-c60c06a9dfbf-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:16.509591 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.509585    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c44p9\" (UniqueName: \"kubernetes.io/projected/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-kube-api-access-c44p9\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:16.509591 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.509595    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/abf20d47-3f33-4a67-afb7-4ccdd7540cbf-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:16.509826 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:16.509603    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k7tbd\" (UniqueName: \"kubernetes.io/projected/702d5aa7-e543-4d31-9304-c60c06a9dfbf-kube-api-access-k7tbd\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:17.056140 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.056105    2580 generic.go:356] "Generic (PLEG): container finished" podID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" containerID="ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c" exitCode=0
Sep 16 14:40:17.056609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.056165    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"
Sep 16 14:40:17.056609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.056198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" event={"ID":"abf20d47-3f33-4a67-afb7-4ccdd7540cbf","Type":"ContainerDied","Data":"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"}
Sep 16 14:40:17.056609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.056277    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt" event={"ID":"abf20d47-3f33-4a67-afb7-4ccdd7540cbf","Type":"ContainerDied","Data":"9be53b5026be129efde1ee38a14e5d4828c256ab7d12a8de18e843c7478bba7e"}
Sep 16 14:40:17.056609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.056309    2580 scope.go:117] "RemoveContainer" containerID="ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"
Sep 16 14:40:17.057688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.057640    2580 generic.go:356] "Generic (PLEG): container finished" podID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" containerID="48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd" exitCode=0
Sep 16 14:40:17.057688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.057673    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" event={"ID":"702d5aa7-e543-4d31-9304-c60c06a9dfbf","Type":"ContainerDied","Data":"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"}
Sep 16 14:40:17.057795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.057713    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"
Sep 16 14:40:17.057795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.057725    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf" event={"ID":"702d5aa7-e543-4d31-9304-c60c06a9dfbf","Type":"ContainerDied","Data":"4a5dbb3e7a74ad93cb30b3042d2df71a5cc18be65de73d24d977ff557b5b5f7b"}
Sep 16 14:40:17.066967 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.066949    2580 scope.go:117] "RemoveContainer" containerID="ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"
Sep 16 14:40:17.067284 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:17.067257    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c\": container with ID starting with ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c not found: ID does not exist" containerID="ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"
Sep 16 14:40:17.067380 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.067295    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c"} err="failed to get container status \"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c\": rpc error: code = NotFound desc = could not find container \"ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c\": container with ID starting with ac2850db856a2c0c9e103e5dc86200e80e7018f6306c217976c01df8ed50a58c not found: ID does not exist"
Sep 16 14:40:17.067380 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.067320    2580 scope.go:117] "RemoveContainer" containerID="48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"
Sep 16 14:40:17.075754 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.075737    2580 scope.go:117] "RemoveContainer" containerID="48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"
Sep 16 14:40:17.076004 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:17.075986    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd\": container with ID starting with 48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd not found: ID does not exist" containerID="48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"
Sep 16 14:40:17.076044 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.076014    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd"} err="failed to get container status \"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd\": rpc error: code = NotFound desc = could not find container \"48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd\": container with ID starting with 48a5987ea8cd7b5b95c62f7313fdddd646ccc104f012aaef59aacd75c9b51cdd not found: ID does not exist"
Sep 16 14:40:17.088401 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.088359    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:40:17.091730 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.091707    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-b-w1-bdfd4a-566b7cdf8c-fbwmt"]
Sep 16 14:40:17.106778 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.106754    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:40:17.117846 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.117819    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w1-bdfd4a-c58b685c8-dzxrf"]
Sep 16 14:40:17.184724 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.184685    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29b49a3a-492c-4617-bd71-cb4af07d025e" path="/var/lib/kubelet/pods/29b49a3a-492c-4617-bd71-cb4af07d025e/volumes"
Sep 16 14:40:17.185195 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.185176    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" path="/var/lib/kubelet/pods/702d5aa7-e543-4d31-9304-c60c06a9dfbf/volumes"
Sep 16 14:40:17.185506 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:17.185495    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" path="/var/lib/kubelet/pods/abf20d47-3f33-4a67-afb7-4ccdd7540cbf/volumes"
Sep 16 14:40:21.373044 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:21.373011    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:40:21.373547 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:21.373307    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" containerID="cri-o://602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1" gracePeriod=30
Sep 16 14:40:24.756923 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:24.756888    2580 patch_prober.go:28] interesting pod/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:40:24.757315 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:24.756961    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:40:24.762958 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:24.762930    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:40:24.763135 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:24.763115    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" containerID="cri-o://219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea" gracePeriod=30
Sep 16 14:40:25.163872 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.163793    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"]
Sep 16 14:40:25.164179 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164166    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164180    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164190    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164196    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164205    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="889b9dd6-3da4-4727-a394-5afef24d3fc5" containerName="payload-pre-processing"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164211    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="889b9dd6-3da4-4727-a394-5afef24d3fc5" containerName="payload-pre-processing"
Sep 16 14:40:25.164218 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164219    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29b49a3a-492c-4617-bd71-cb4af07d025e" containerName="revoke-keys"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164225    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29b49a3a-492c-4617-bd71-cb4af07d025e" containerName="revoke-keys"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164240    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" containerName="payload-pre-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164245    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" containerName="payload-pre-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164251    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" containerName="payload-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164256    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" containerName="payload-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164316    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="702d5aa7-e543-4d31-9304-c60c06a9dfbf" containerName="payload-pre-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164326    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164334    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="d96c0af5-b2f9-44a6-9853-0ccf1ead036f" containerName="istio-proxy"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164340    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29b49a3a-492c-4617-bd71-cb4af07d025e" containerName="revoke-keys"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164346    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="abf20d47-3f33-4a67-afb7-4ccdd7540cbf" containerName="payload-processing"
Sep 16 14:40:25.164411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.164354    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="889b9dd6-3da4-4727-a394-5afef24d3fc5" containerName="payload-pre-processing"
Sep 16 14:40:25.167239 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.167221    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:25.172624 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.172599    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:40:25.172817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.172797    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:40:25.185584 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.185537    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"]
Sep 16 14:40:25.298181 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.298148    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw\" (UID: \"86d2f57a-3021-48c9-91dc-0519b2b6a99b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:25.399701 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.399670    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw\" (UID: \"86d2f57a-3021-48c9-91dc-0519b2b6a99b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:25.400267 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.400247    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw\" (UID: \"86d2f57a-3021-48c9-91dc-0519b2b6a99b\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:25.477269 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.477238    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:25.792349 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:25.792325    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"]
Sep 16 14:40:25.794285 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:40:25.794259    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86d2f57a_3021_48c9_91dc_0519b2b6a99b.slice/crio-f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e WatchSource:0}: Error finding container f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e: Status 404 returned error can't find the container with id f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e
Sep 16 14:40:26.109598 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.109545    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw" event={"ID":"86d2f57a-3021-48c9-91dc-0519b2b6a99b","Type":"ContainerStarted","Data":"7f3e5d04654f837a03259dede5ea3e46014d58060cb086dfdf5ef19859a955f8"}
Sep 16 14:40:26.109745 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.109623    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw" event={"ID":"86d2f57a-3021-48c9-91dc-0519b2b6a99b","Type":"ContainerStarted","Data":"f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e"}
Sep 16 14:40:26.135465 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.135416    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw" podStartSLOduration=1.135400984 podStartE2EDuration="1.135400984s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:40:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:40:26.133080713 +0000 UTC m=+1947.539660922" watchObservedRunningTime="2026-09-16 14:40:26.135400984 +0000 UTC m=+1947.541981232"
Sep 16 14:40:26.660029 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.660003    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:26.784275 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.784236    2580 patch_prober.go:28] interesting pod/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 16 14:40:26.784443 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.784299    2580 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 16 14:40:26.814961 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.814929    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.814974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.814997    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815063    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815080    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815146    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dg4x9\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815189    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815216    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815243    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs\") pod \"ccb21e37-f739-41e8-8305-65f13cd6896b\" (UID: \"ccb21e37-f739-41e8-8305-65f13cd6896b\") "
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815264    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:26.815358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815341    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data" (OuterVolumeSpecName: "istio-data") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815544    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815634    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815653    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815670    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815709    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:26.815850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.815753    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:26.817420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.817401    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9" (OuterVolumeSpecName: "kube-api-access-dg4x9") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "kube-api-access-dg4x9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:26.817519 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.817502    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token" (OuterVolumeSpecName: "istio-token") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:26.817654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.817638    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:26.817654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.817643    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "ccb21e37-f739-41e8-8305-65f13cd6896b" (UID: "ccb21e37-f739-41e8-8305-65f13cd6896b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916657    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916702    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dg4x9\" (UniqueName: \"kubernetes.io/projected/ccb21e37-f739-41e8-8305-65f13cd6896b-kube-api-access-dg4x9\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916718    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ccb21e37-f739-41e8-8305-65f13cd6896b-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916729    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916741    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:26.916767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:26.916751    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ccb21e37-f739-41e8-8305-65f13cd6896b-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:27.116306 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.116272    2580 generic.go:356] "Generic (PLEG): container finished" podID="86d2f57a-3021-48c9-91dc-0519b2b6a99b" containerID="7f3e5d04654f837a03259dede5ea3e46014d58060cb086dfdf5ef19859a955f8" exitCode=0
Sep 16 14:40:27.116489 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.116366    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw" event={"ID":"86d2f57a-3021-48c9-91dc-0519b2b6a99b","Type":"ContainerDied","Data":"7f3e5d04654f837a03259dede5ea3e46014d58060cb086dfdf5ef19859a955f8"}
Sep 16 14:40:27.118068 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.118045    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerID="602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1" exitCode=0
Sep 16 14:40:27.118184 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.118079    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"
Sep 16 14:40:27.118184 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.118081    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerDied","Data":"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"}
Sep 16 14:40:27.118264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.118223    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9" event={"ID":"ccb21e37-f739-41e8-8305-65f13cd6896b","Type":"ContainerDied","Data":"a81af0531f3b5c435a3317e44a992c6d4a72e6b304d1b8d6ac303af0193b7904"}
Sep 16 14:40:27.118264 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.118254    2580 scope.go:117] "RemoveContainer" containerID="602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"
Sep 16 14:40:27.133240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.133219    2580 scope.go:117] "RemoveContainer" containerID="f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"
Sep 16 14:40:27.141850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.141829    2580 scope.go:117] "RemoveContainer" containerID="602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"
Sep 16 14:40:27.142122 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:27.142103    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1\": container with ID starting with 602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1 not found: ID does not exist" containerID="602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"
Sep 16 14:40:27.142185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.142130    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1"} err="failed to get container status \"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1\": rpc error: code = NotFound desc = could not find container \"602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1\": container with ID starting with 602d6b3962d4f80fff461cc549a7037c2af59f08039af24d10f80f7f0f4226d1 not found: ID does not exist"
Sep 16 14:40:27.142185 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.142145    2580 scope.go:117] "RemoveContainer" containerID="f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"
Sep 16 14:40:27.142396 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:27.142379    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977\": container with ID starting with f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977 not found: ID does not exist" containerID="f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"
Sep 16 14:40:27.142449 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.142406    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977"} err="failed to get container status \"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977\": rpc error: code = NotFound desc = could not find container \"f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977\": container with ID starting with f7dec9e06f298b4c75db06da6f8a2475600c46fa335e529a013f9cb236635977 not found: ID does not exist"
Sep 16 14:40:27.164365 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.164341    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:40:27.166246 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.166225    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-cleanup-d5498f-openshift-default-55566dfb-xhfs9"]
Sep 16 14:40:27.185676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:27.185605    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" path="/var/lib/kubelet/pods/ccb21e37-f739-41e8-8305-65f13cd6896b/volumes"
Sep 16 14:40:28.255326 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:28.255303    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:28.329727 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:28.329697    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca\") pod \"86d2f57a-3021-48c9-91dc-0519b2b6a99b\" (UID: \"86d2f57a-3021-48c9-91dc-0519b2b6a99b\") "
Sep 16 14:40:28.330012 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:28.329991    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "86d2f57a-3021-48c9-91dc-0519b2b6a99b" (UID: "86d2f57a-3021-48c9-91dc-0519b2b6a99b"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:28.431527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:28.431451    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/86d2f57a-3021-48c9-91dc-0519b2b6a99b-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:29.131516 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:29.131486    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"
Sep 16 14:40:29.131721 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:29.131482    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw" event={"ID":"86d2f57a-3021-48c9-91dc-0519b2b6a99b","Type":"ContainerDied","Data":"f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e"}
Sep 16 14:40:29.131721 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:29.131599    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f41b9ec81d0b2c082249fa47d715f86dbebdf06e6e7eeb778f62543213757c7e"
Sep 16 14:40:30.026494 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.026419    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:40:30.138155 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.138121    2580 generic.go:356] "Generic (PLEG): container finished" podID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerID="219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea" exitCode=0
Sep 16 14:40:30.138329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.138170    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"
Sep 16 14:40:30.138329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.138205    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerDied","Data":"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"}
Sep 16 14:40:30.138329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.138271    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5" event={"ID":"1e80ecc0-9e86-4f16-8854-280fce70bc76","Type":"ContainerDied","Data":"ecc1cd30dd4cc4cb82a16fa5c17599c0aa57edc0408a63c2996a6b1558f2d09e"}
Sep 16 14:40:30.138329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.138297    2580 scope.go:117] "RemoveContainer" containerID="219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"
Sep 16 14:40:30.147168 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.147149    2580 scope.go:117] "RemoveContainer" containerID="094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"
Sep 16 14:40:30.149371 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149344    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149414    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149453    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149488 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149484    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149600    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149653    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149684 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149602    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:30.149817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149687    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149719    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zfpfq\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149817 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149797    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket\") pod \"1e80ecc0-9e86-4f16-8854-280fce70bc76\" (UID: \"1e80ecc0-9e86-4f16-8854-280fce70bc76\") "
Sep 16 14:40:30.149980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149926    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data" (OuterVolumeSpecName: "istio-data") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:30.149980 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.149957    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:30.150120 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150095    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:30.150198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150161    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:30.150257 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150232    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.150314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150253    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1e80ecc0-9e86-4f16-8854-280fce70bc76-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.150314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150268    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.150314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150281    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.150314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.150293    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.151808 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.151785    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token" (OuterVolumeSpecName: "istio-token") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:30.151918 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.151899    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq" (OuterVolumeSpecName: "kube-api-access-zfpfq") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "kube-api-access-zfpfq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:30.152100 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.152077    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:40:30.152201 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.152187    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1e80ecc0-9e86-4f16-8854-280fce70bc76" (UID: "1e80ecc0-9e86-4f16-8854-280fce70bc76"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:40:30.156134 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.156115    2580 scope.go:117] "RemoveContainer" containerID="219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"
Sep 16 14:40:30.156452 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:30.156434    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea\": container with ID starting with 219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea not found: ID does not exist" containerID="219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"
Sep 16 14:40:30.156510 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.156460    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea"} err="failed to get container status \"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea\": rpc error: code = NotFound desc = could not find container \"219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea\": container with ID starting with 219d3e287e10eddcb7e74facaac18ce885c87ed3d88977f4144101d4e556f7ea not found: ID does not exist"
Sep 16 14:40:30.156510 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.156478    2580 scope.go:117] "RemoveContainer" containerID="094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"
Sep 16 14:40:30.156760 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:30.156744    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf\": container with ID starting with 094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf not found: ID does not exist" containerID="094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"
Sep 16 14:40:30.156812 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.156771    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf"} err="failed to get container status \"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf\": rpc error: code = NotFound desc = could not find container \"094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf\": container with ID starting with 094ad4abce61751ea7802025d8aca0623778b328bdd4615d6c7cf1347f4dd8bf not found: ID does not exist"
Sep 16 14:40:30.251070 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.251025    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.251070 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.251063    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.251070 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.251072    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zfpfq\" (UniqueName: \"kubernetes.io/projected/1e80ecc0-9e86-4f16-8854-280fce70bc76-kube-api-access-zfpfq\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.251307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.251084    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1e80ecc0-9e86-4f16-8854-280fce70bc76-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:30.468425 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.468393    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:40:30.476234 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:30.476196    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-bdfd4a-openshift-default-6cd8b8f476-28zh5"]
Sep 16 14:40:31.184682 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:31.184647    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" path="/var/lib/kubelet/pods/1e80ecc0-9e86-4f16-8854-280fce70bc76/volumes"
Sep 16 14:40:40.875901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:40.875869    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:40:40.876361 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:40.876087    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" podUID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" containerName="payload-pre-processing" containerID="cri-o://99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa" gracePeriod=30
Sep 16 14:40:41.121620 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.121594    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:40:41.201127 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.201092    2580 generic.go:356] "Generic (PLEG): container finished" podID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" containerID="99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa" exitCode=0
Sep 16 14:40:41.201274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.201142    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"
Sep 16 14:40:41.201274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.201161    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" event={"ID":"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac","Type":"ContainerDied","Data":"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"}
Sep 16 14:40:41.201274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.201201    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd" event={"ID":"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac","Type":"ContainerDied","Data":"bca44a31b4a2d784062d95a6f4ba3a1de8af3c56c200c42a0b6e16ab7602b1c7"}
Sep 16 14:40:41.201274 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.201224    2580 scope.go:117] "RemoveContainer" containerID="99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"
Sep 16 14:40:41.209655 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.209636    2580 scope.go:117] "RemoveContainer" containerID="99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"
Sep 16 14:40:41.209955 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:40:41.209928    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa\": container with ID starting with 99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa not found: ID does not exist" containerID="99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"
Sep 16 14:40:41.210090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.209962    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa"} err="failed to get container status \"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa\": rpc error: code = NotFound desc = could not find container \"99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa\": container with ID starting with 99c43f9f3d1a4391164eaa415d28ebf022f7b48fa0f861e58412b5001b19c1fa not found: ID does not exist"
Sep 16 14:40:41.259832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.259794    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"]
Sep 16 14:40:41.260019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.259871    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ggv77\" (UniqueName: \"kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77\") pod \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") "
Sep 16 14:40:41.260019 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.259933    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume\") pod \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\" (UID: \"ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac\") "
Sep 16 14:40:41.260713 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.260678    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" (UID: "ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:40:41.263253 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.263201    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-363d71-s8ctw"]
Sep 16 14:40:41.263466 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.263440    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77" (OuterVolumeSpecName: "kube-api-access-ggv77") pod "ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" (UID: "ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac"). InnerVolumeSpecName "kube-api-access-ggv77". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:40:41.361454 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.361409    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ggv77\" (UniqueName: \"kubernetes.io/projected/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-kube-api-access-ggv77\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:41.361454 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.361456    2580 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac-plugins-config-volume\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:40:41.524927 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.524898    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:40:41.531317 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:41.531290    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-363d71-566994f74-9q9xd"]
Sep 16 14:40:43.179062 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:43.179027    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:40:43.184571 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:43.184526    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86d2f57a-3021-48c9-91dc-0519b2b6a99b" path="/var/lib/kubelet/pods/86d2f57a-3021-48c9-91dc-0519b2b6a99b/volumes"
Sep 16 14:40:43.184915 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:40:43.184901    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" path="/var/lib/kubelet/pods/ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac/volumes"
Sep 16 14:41:45.120694 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.120662    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:41:45.121636 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.120902    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy" containerID="cri-o://5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2" gracePeriod=30
Sep 16 14:41:45.502314 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502278    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"]
Sep 16 14:41:45.502683 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502670    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502684    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502693    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86d2f57a-3021-48c9-91dc-0519b2b6a99b" containerName="revoke-keys"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502699    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="86d2f57a-3021-48c9-91dc-0519b2b6a99b" containerName="revoke-keys"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502711    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502716    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502726    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502731    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502739    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.502741 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502744    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502753    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" containerName="payload-pre-processing"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502758    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" containerName="payload-pre-processing"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502811    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502817    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502824    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccc9df55-2381-44f7-9f2a-85f5ecb7d1ac" containerName="payload-pre-processing"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502832    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ccb21e37-f739-41e8-8305-65f13cd6896b" containerName="istio-proxy"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502838    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="86d2f57a-3021-48c9-91dc-0519b2b6a99b" containerName="revoke-keys"
Sep 16 14:41:45.503022 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.502844    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e80ecc0-9e86-4f16-8854-280fce70bc76" containerName="istio-proxy"
Sep 16 14:41:45.505832 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.505816    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:45.509112 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.509086    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:41:45.509215 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.509160    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:41:45.521216 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.519331    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"]
Sep 16 14:41:45.673211 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.673172    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw\" (UID: \"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:45.774054 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.773971    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw\" (UID: \"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:45.774612 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.774590    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw\" (UID: \"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:45.817383 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:45.817353    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:46.112654 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:46.112626    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"]
Sep 16 14:41:46.115130 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:41:46.115104    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8ccc439d_e65c_4fc5_ad9f_cb6cb779b4e3.slice/crio-f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8 WatchSource:0}: Error finding container f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8: Status 404 returned error can't find the container with id f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8
Sep 16 14:41:46.530240 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:46.530198    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw" event={"ID":"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3","Type":"ContainerStarted","Data":"00dc1de4164d5110fa8fcaae4354f409dd10a6dd50723ed54391681eb018549a"}
Sep 16 14:41:46.530652 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:46.530255    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw" event={"ID":"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3","Type":"ContainerStarted","Data":"f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8"}
Sep 16 14:41:46.548301 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:46.548256    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw" podStartSLOduration=1.548237725 podStartE2EDuration="1.548237725s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:41:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:41:46.547706184 +0000 UTC m=+2027.954286418" watchObservedRunningTime="2026-09-16 14:41:46.548237725 +0000 UTC m=+2027.954817929"
Sep 16 14:41:47.536583 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:47.536524    2580 generic.go:356] "Generic (PLEG): container finished" podID="8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" containerID="00dc1de4164d5110fa8fcaae4354f409dd10a6dd50723ed54391681eb018549a" exitCode=0
Sep 16 14:41:47.536988 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:47.536611    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw" event={"ID":"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3","Type":"ContainerDied","Data":"00dc1de4164d5110fa8fcaae4354f409dd10a6dd50723ed54391681eb018549a"}
Sep 16 14:41:48.673954 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:48.673930    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:48.805667 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:48.805537    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca\") pod \"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3\" (UID: \"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3\") "
Sep 16 14:41:48.805956 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:48.805933    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" (UID: "8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:41:48.906633 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:48.906597    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:49.548172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:49.548083    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"
Sep 16 14:41:49.548172 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:49.548125    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw" event={"ID":"8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3","Type":"ContainerDied","Data":"f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8"}
Sep 16 14:41:49.548372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:49.548183    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f820ed2f612f1988fd6dccca554959744474eaf81fb7e6dff6f45e05edd44aa8"
Sep 16 14:41:50.178710 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.178684    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:41:50.362058 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.362035    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:41:50.522095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522062    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tbcjn\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522097    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522124    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522140    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522160    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522230    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522266    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522363 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522338    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522392    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs\") pod \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\" (UID: \"bef0cc4c-997f-430c-b9d1-2c3ac0576563\") "
Sep 16 14:41:50.522728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522436    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:50.522728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522657    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:50.522728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522665    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:41:50.522945 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522788    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.522945 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522836    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.522945 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522851    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.522945 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522854    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:50.523161 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.522948    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data" (OuterVolumeSpecName: "istio-data") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:50.524386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.524357    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn" (OuterVolumeSpecName: "kube-api-access-tbcjn") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "kube-api-access-tbcjn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:41:50.524386 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.524365    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:41:50.524539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.524446    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token" (OuterVolumeSpecName: "istio-token") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:41:50.524539 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.524453    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bef0cc4c-997f-430c-b9d1-2c3ac0576563" (UID: "bef0cc4c-997f-430c-b9d1-2c3ac0576563"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:41:50.554204 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.554173    2580 generic.go:356] "Generic (PLEG): container finished" podID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerID="5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2" exitCode=0
Sep 16 14:41:50.554351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.554226    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"
Sep 16 14:41:50.554351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.554259    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerDied","Data":"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"}
Sep 16 14:41:50.554351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.554316    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j" event={"ID":"bef0cc4c-997f-430c-b9d1-2c3ac0576563","Type":"ContainerDied","Data":"83ee072868e5a0fa512302b0134be3faff28cf71fe42e110e0f3094e3524ca7d"}
Sep 16 14:41:50.554351 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.554342    2580 scope.go:117] "RemoveContainer" containerID="5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"
Sep 16 14:41:50.569714 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.569692    2580 scope.go:117] "RemoveContainer" containerID="c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"
Sep 16 14:41:50.577495 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.577475    2580 scope.go:117] "RemoveContainer" containerID="5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"
Sep 16 14:41:50.577758 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:41:50.577740    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2\": container with ID starting with 5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2 not found: ID does not exist" containerID="5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"
Sep 16 14:41:50.577811 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.577767    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2"} err="failed to get container status \"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2\": rpc error: code = NotFound desc = could not find container \"5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2\": container with ID starting with 5690978bb789f76fbe0969d1c2fe5f520b9eb1b30a762604b8ff639e80341db2 not found: ID does not exist"
Sep 16 14:41:50.577811 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.577784    2580 scope.go:117] "RemoveContainer" containerID="c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"
Sep 16 14:41:50.578014 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:41:50.577993    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8\": container with ID starting with c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8 not found: ID does not exist" containerID="c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"
Sep 16 14:41:50.578075 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.578025    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8"} err="failed to get container status \"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8\": rpc error: code = NotFound desc = could not find container \"c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8\": container with ID starting with c7cbb18e4860e15e5a6a0cd324968ee177c04762f0a562f43f3ce8a1ff9fd0c8 not found: ID does not exist"
Sep 16 14:41:50.581704 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.581684    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:41:50.585663 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.585645    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-363d71-openshift-default-76c8f79f9d-57h9j"]
Sep 16 14:41:50.624253 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624225    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.624253 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624250    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tbcjn\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-kube-api-access-tbcjn\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.624366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624260    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.624366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624270    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.624366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624278    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:50.624366 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:50.624285    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bef0cc4c-997f-430c-b9d1-2c3ac0576563-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:41:51.183801 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:51.183766    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" path="/var/lib/kubelet/pods/bef0cc4c-997f-430c-b9d1-2c3ac0576563/volumes"
Sep 16 14:41:56.395434 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:56.395389    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"]
Sep 16 14:41:56.398678 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:56.398647    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-5f0bee-lgfhw"]
Sep 16 14:41:57.183795 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:57.183763    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" path="/var/lib/kubelet/pods/8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3/volumes"
Sep 16 14:41:59.444047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:41:59.444005    2580 scope.go:117] "RemoveContainer" containerID="92c6f8d537832d22fce87cd67caaa2583127eab9932b08e5552261eedcfe8f90"
Sep 16 14:42:03.321903 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.321868    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:42:03.322413 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.322088    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy" containerID="cri-o://f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63" gracePeriod=30
Sep 16 14:42:03.660037 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.659944    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"]
Sep 16 14:42:03.660411 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660392    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:03.660497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660414    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:03.660497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660434    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:03.660497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660442    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:03.660497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660458    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" containerName="revoke-keys"
Sep 16 14:42:03.660497 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660467    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" containerName="revoke-keys"
Sep 16 14:42:03.660860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660608    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:03.660860 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.660622    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8ccc439d-e65c-4fc5-ad9f-cb6cb779b4e3" containerName="revoke-keys"
Sep 16 14:42:03.663779 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.663761    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:03.666527 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.666505    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:42:03.666699 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.666539    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:42:03.672210 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.672190    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"]
Sep 16 14:42:03.748090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.748047    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4\" (UID: \"1fc1e0c8-687d-49d7-a489-8fc68c657679\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:03.849193 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.849149    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4\" (UID: \"1fc1e0c8-687d-49d7-a489-8fc68c657679\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:03.849837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.849816    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4\" (UID: \"1fc1e0c8-687d-49d7-a489-8fc68c657679\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:03.975597 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:03.975537    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:04.253588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:04.253536    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"]
Sep 16 14:42:04.256129 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:42:04.256098    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc1e0c8_687d_49d7_a489_8fc68c657679.slice/crio-2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e WatchSource:0}: Error finding container 2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e: Status 404 returned error can't find the container with id 2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e
Sep 16 14:42:04.623422 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:04.623389    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4" event={"ID":"1fc1e0c8-687d-49d7-a489-8fc68c657679","Type":"ContainerStarted","Data":"55979a0c6cb45fb5b7dee73fbd1f96fd6fda99162e48b219d98974e410bf5fc0"}
Sep 16 14:42:04.623876 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:04.623461    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4" event={"ID":"1fc1e0c8-687d-49d7-a489-8fc68c657679","Type":"ContainerStarted","Data":"2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e"}
Sep 16 14:42:04.644794 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:04.644741    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4" podStartSLOduration=1.644723086 podStartE2EDuration="1.644723086s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:42:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:42:04.642274532 +0000 UTC m=+2046.048854754" watchObservedRunningTime="2026-09-16 14:42:04.644723086 +0000 UTC m=+2046.051303294"
Sep 16 14:42:05.629670 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:05.629630    2580 generic.go:356] "Generic (PLEG): container finished" podID="1fc1e0c8-687d-49d7-a489-8fc68c657679" containerID="55979a0c6cb45fb5b7dee73fbd1f96fd6fda99162e48b219d98974e410bf5fc0" exitCode=0
Sep 16 14:42:05.630128 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:05.629681    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4" event={"ID":"1fc1e0c8-687d-49d7-a489-8fc68c657679","Type":"ContainerDied","Data":"55979a0c6cb45fb5b7dee73fbd1f96fd6fda99162e48b219d98974e410bf5fc0"}
Sep 16 14:42:06.770887 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:06.770860    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:06.878003 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:06.877960    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca\") pod \"1fc1e0c8-687d-49d7-a489-8fc68c657679\" (UID: \"1fc1e0c8-687d-49d7-a489-8fc68c657679\") "
Sep 16 14:42:06.878323 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:06.878294    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1fc1e0c8-687d-49d7-a489-8fc68c657679" (UID: "1fc1e0c8-687d-49d7-a489-8fc68c657679"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:06.979145 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:06.979104    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1fc1e0c8-687d-49d7-a489-8fc68c657679-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:07.641083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:07.641051    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"
Sep 16 14:42:07.641083 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:07.641061    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4" event={"ID":"1fc1e0c8-687d-49d7-a489-8fc68c657679","Type":"ContainerDied","Data":"2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e"}
Sep 16 14:42:07.641358 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:07.641116    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2a3eab1091d9ab64f67716f7eb4aea3cfdd4edf48e0536a1428f3eefa887741e"
Sep 16 14:42:08.575750 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.575723    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:42:08.648937 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.648849    2580 generic.go:356] "Generic (PLEG): container finished" podID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerID="f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63" exitCode=0
Sep 16 14:42:08.648937 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.648901    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"
Sep 16 14:42:08.649131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.648938    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerDied","Data":"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"}
Sep 16 14:42:08.649131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.649001    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz" event={"ID":"5eacbbd1-2f77-4095-aac9-6387699aa6f1","Type":"ContainerDied","Data":"bcd46166dd19f7ff9851614344cebed767f923132eb5d789408f3f7c2dac6b68"}
Sep 16 14:42:08.649131 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.649032    2580 scope.go:117] "RemoveContainer" containerID="f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"
Sep 16 14:42:08.657715 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.657692    2580 scope.go:117] "RemoveContainer" containerID="e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"
Sep 16 14:42:08.665427 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.665406    2580 scope.go:117] "RemoveContainer" containerID="f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"
Sep 16 14:42:08.665713 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:42:08.665691    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63\": container with ID starting with f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63 not found: ID does not exist" containerID="f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"
Sep 16 14:42:08.665807 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.665722    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63"} err="failed to get container status \"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63\": rpc error: code = NotFound desc = could not find container \"f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63\": container with ID starting with f2fff5583c73748ea00c86411ffc9e5ed549b125eeb2fd389b7eadc63ec44c63 not found: ID does not exist"
Sep 16 14:42:08.665807 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.665741    2580 scope.go:117] "RemoveContainer" containerID="e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"
Sep 16 14:42:08.666002 ip-10-0-138-80 kubenswrapper[2580]: E0916 14:42:08.665984    2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4\": container with ID starting with e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4 not found: ID does not exist" containerID="e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"
Sep 16 14:42:08.666047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.666010    2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4"} err="failed to get container status \"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4\": rpc error: code = NotFound desc = could not find container \"e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4\": container with ID starting with e968d58344f99d1a1ee5214176a1632c183cb8031f9bb497317140773a67fbd4 not found: ID does not exist"
Sep 16 14:42:08.696399 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696371    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696408    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696463    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696485    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8x4kb\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696512 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696503    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696523    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696646    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:08.696738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696698    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.696738 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696731    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket\") pod \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\" (UID: \"5eacbbd1-2f77-4095-aac9-6387699aa6f1\") "
Sep 16 14:42:08.697000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696836    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data" (OuterVolumeSpecName: "istio-data") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:08.697000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.696922    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:08.697121 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697045    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:08.697214 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697186    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:08.697277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697253    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.697277 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697275    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.697387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697288    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.697387 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.697301    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.698892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.698850    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:42:08.698892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.698874    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb" (OuterVolumeSpecName: "kube-api-access-8x4kb") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "kube-api-access-8x4kb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:08.698892 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.698862    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token" (OuterVolumeSpecName: "istio-token") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:08.699065 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.698925    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5eacbbd1-2f77-4095-aac9-6387699aa6f1" (UID: "5eacbbd1-2f77-4095-aac9-6387699aa6f1"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:08.798609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.798540    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.798609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.798601    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5eacbbd1-2f77-4095-aac9-6387699aa6f1-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.798609 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.798614    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8x4kb\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-kube-api-access-8x4kb\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.798855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.798627    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.798855 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.798638    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5eacbbd1-2f77-4095-aac9-6387699aa6f1-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:08.974808 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.974775    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:42:08.980027 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:08.979999    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-5f0bee-openshift-default-965997d4b-bjlwz"]
Sep 16 14:42:09.184247 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:09.184216    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" path="/var/lib/kubelet/pods/5eacbbd1-2f77-4095-aac9-6387699aa6f1/volumes"
Sep 16 14:42:14.366829 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:14.366785    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"]
Sep 16 14:42:14.379047 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:14.379015    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-c0974e-29826150-9w5qp"]
Sep 16 14:42:14.775021 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:14.774983    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"]
Sep 16 14:42:14.782983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:14.782955    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-c0974e-xfzp4"]
Sep 16 14:42:15.183688 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:15.183609    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc1e0c8-687d-49d7-a489-8fc68c657679" path="/var/lib/kubelet/pods/1fc1e0c8-687d-49d7-a489-8fc68c657679/volumes"
Sep 16 14:42:15.183930 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:15.183912    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d03659d-3de2-4cd1-b426-ac79ae672f33" path="/var/lib/kubelet/pods/7d03659d-3de2-4cd1-b426-ac79ae672f33/volumes"
Sep 16 14:42:22.685419 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685337    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"]
Sep 16 14:42:22.685819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685752    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc1e0c8-687d-49d7-a489-8fc68c657679" containerName="revoke-keys"
Sep 16 14:42:22.685819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685767    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc1e0c8-687d-49d7-a489-8fc68c657679" containerName="revoke-keys"
Sep 16 14:42:22.685819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685780    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:42:22.685819 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685786    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:42:22.685955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685850    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:42:22.685955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685860    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:42:22.685955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685867    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bef0cc4c-997f-430c-b9d1-2c3ac0576563" containerName="istio-proxy"
Sep 16 14:42:22.685955 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.685874    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1fc1e0c8-687d-49d7-a489-8fc68c657679" containerName="revoke-keys"
Sep 16 14:42:22.690279 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.690261    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:22.693095 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.693075    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:42:22.693232 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.693150    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 16 14:42:22.716926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.716893    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"]
Sep 16 14:42:22.828865 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.828831    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz\" (UID: \"20b96c6f-5f11-411d-837e-87e9f3a5b596\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:22.929728 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.929684    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz\" (UID: \"20b96c6f-5f11-411d-837e-87e9f3a5b596\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:22.930308 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.930286    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz\" (UID: \"20b96c6f-5f11-411d-837e-87e9f3a5b596\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:22.999603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:22.999571    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:23.277798 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:23.277772    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"]
Sep 16 14:42:23.279682 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:42:23.279654    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20b96c6f_5f11_411d_837e_87e9f3a5b596.slice/crio-97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04 WatchSource:0}: Error finding container 97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04: Status 404 returned error can't find the container with id 97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04
Sep 16 14:42:23.724702 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:23.724659    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz" event={"ID":"20b96c6f-5f11-411d-837e-87e9f3a5b596","Type":"ContainerStarted","Data":"2cb01d45dd3ae3d03d900a86fab8c07d5584e180e346408e18ef84e498728ae8"}
Sep 16 14:42:23.725090 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:23.724722    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz" event={"ID":"20b96c6f-5f11-411d-837e-87e9f3a5b596","Type":"ContainerStarted","Data":"97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04"}
Sep 16 14:42:23.748475 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:23.748430    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz" podStartSLOduration=1.748411796 podStartE2EDuration="1.748411796s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-16 14:42:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:42:23.748003937 +0000 UTC m=+2065.154584144" watchObservedRunningTime="2026-09-16 14:42:23.748411796 +0000 UTC m=+2065.154992005"
Sep 16 14:42:24.731565 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:24.731532    2580 generic.go:356] "Generic (PLEG): container finished" podID="20b96c6f-5f11-411d-837e-87e9f3a5b596" containerID="2cb01d45dd3ae3d03d900a86fab8c07d5584e180e346408e18ef84e498728ae8" exitCode=0
Sep 16 14:42:24.732041 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:24.731617    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz" event={"ID":"20b96c6f-5f11-411d-837e-87e9f3a5b596","Type":"ContainerDied","Data":"2cb01d45dd3ae3d03d900a86fab8c07d5584e180e346408e18ef84e498728ae8"}
Sep 16 14:42:25.866875 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:25.866852    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:26.060068 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.059974    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca\") pod \"20b96c6f-5f11-411d-837e-87e9f3a5b596\" (UID: \"20b96c6f-5f11-411d-837e-87e9f3a5b596\") "
Sep 16 14:42:26.060364 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.060340    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "20b96c6f-5f11-411d-837e-87e9f3a5b596" (UID: "20b96c6f-5f11-411d-837e-87e9f3a5b596"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:26.161086 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.161050    2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/20b96c6f-5f11-411d-837e-87e9f3a5b596-maas-api-service-ca\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:26.742686 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.742658    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"
Sep 16 14:42:26.742869 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.742697    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz" event={"ID":"20b96c6f-5f11-411d-837e-87e9f3a5b596","Type":"ContainerDied","Data":"97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04"}
Sep 16 14:42:26.742869 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:26.742740    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="97b1bf1c00d0fcfbdd851578faf94ca1f4cbcbf355b9d85b3bbe0e483625ff04"
Sep 16 14:42:33.489790 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:33.489757    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"]
Sep 16 14:42:33.496224 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:33.496198    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w2-d31d7c-29826150-2thtx"]
Sep 16 14:42:33.746747 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:33.746661    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"]
Sep 16 14:42:33.752350 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:33.752320    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-d31d7c-rjsvz"]
Sep 16 14:42:35.183622 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:35.183585    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20b96c6f-5f11-411d-837e-87e9f3a5b596" path="/var/lib/kubelet/pods/20b96c6f-5f11-411d-837e-87e9f3a5b596/volumes"
Sep 16 14:42:35.184000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:35.183906    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ae1eab06-146f-4f0a-bb28-bb2e6971ada2" path="/var/lib/kubelet/pods/ae1eab06-146f-4f0a-bb28-bb2e6971ada2/volumes"
Sep 16 14:42:45.691114 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:45.691081    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:42:45.691651 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:45.691304    2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy" containerID="cri-o://77fd84e6bbab43f150672101e528dfe115cbc2ac6fd49d9ac9da45d6cfa7f9e7" gracePeriod=30
Sep 16 14:42:50.871977 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.871941    2580 generic.go:356] "Generic (PLEG): container finished" podID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerID="77fd84e6bbab43f150672101e528dfe115cbc2ac6fd49d9ac9da45d6cfa7f9e7" exitCode=0
Sep 16 14:42:50.872597 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.872004    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerDied","Data":"77fd84e6bbab43f150672101e528dfe115cbc2ac6fd49d9ac9da45d6cfa7f9e7"}
Sep 16 14:42:50.872597 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.872069    2580 scope.go:117] "RemoveContainer" containerID="da76d8ed25815c9285cebe2c5d61b62c3a6a0ac6bc2b8ff61b20cfaa770607f7"
Sep 16 14:42:50.936471 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.936446    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:42:50.992382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992299    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992340    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7wqhd\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992382 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992380    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992411    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992433    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992464    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992493    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992516    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992578    2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy\") pod \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\" (UID: \"f5b6052c-f79f-4707-8dda-60704d2bd6a6\") "
Sep 16 14:42:50.992926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992670    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:50.992926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992731    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:50.992926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992766    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:50.992926 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992797    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 16 14:42:50.993156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992926    2580 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-certs\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:50.993156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992944    2580 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-data\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:50.993156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992956    2580 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istiod-ca-cert\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:50.993156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992960    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:50.993156 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.992966    2580 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-credential-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:50.994859 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.994835    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:50.994964 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.994861    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd" (OuterVolumeSpecName: "kube-api-access-7wqhd") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "kube-api-access-7wqhd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 16 14:42:50.995157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.995134    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 16 14:42:50.995157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:50.995142    2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "f5b6052c-f79f-4707-8dda-60704d2bd6a6" (UID: "f5b6052c-f79f-4707-8dda-60704d2bd6a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 16 14:42:51.093446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.093416    2580 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-podinfo\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:51.093446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.093444    2580 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-workload-socket\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:51.093446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.093453    2580 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-envoy\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:51.093705 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.093462    2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7wqhd\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-kube-api-access-7wqhd\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:51.093705 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.093473    2580 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f5b6052c-f79f-4707-8dda-60704d2bd6a6-istio-token\") on node \"ip-10-0-138-80.ec2.internal\" DevicePath \"\""
Sep 16 14:42:51.878262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.878229    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"
Sep 16 14:42:51.878262 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.878245    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl" event={"ID":"f5b6052c-f79f-4707-8dda-60704d2bd6a6","Type":"ContainerDied","Data":"95f5fcf476eadc7e022f884f4487a8d7f768fc0611639bdbdc1cd3e1de57a678"}
Sep 16 14:42:51.878834 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.878307    2580 scope.go:117] "RemoveContainer" containerID="77fd84e6bbab43f150672101e528dfe115cbc2ac6fd49d9ac9da45d6cfa7f9e7"
Sep 16 14:42:51.896946 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.896921    2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:42:51.906374 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:51.906352    2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w2-d31d7c-openshift-default-588476496-gfvxl"]
Sep 16 14:42:52.178806 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:52.178723    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:42:53.183760 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:53.183725    2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" path="/var/lib/kubelet/pods/f5b6052c-f79f-4707-8dda-60704d2bd6a6/volumes"
Sep 16 14:42:59.467445 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:59.467404    2580 scope.go:117] "RemoveContainer" containerID="09998e15862f9088733cb18db2b3bdd67326a31207977c3227082d1d259f364d"
Sep 16 14:42:59.475841 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:42:59.475818    2580 scope.go:117] "RemoveContainer" containerID="1fff9fba5661a4d5f1eba92e2a6f51483a5d6c4d4eb2af31d285db11a50bf8ed"
Sep 16 14:44:03.178601 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:44:03.178482    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:45:00.138687 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.138651    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"]
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139002    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139012    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139031    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20b96c6f-5f11-411d-837e-87e9f3a5b596" containerName="revoke-keys"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139036    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="20b96c6f-5f11-411d-837e-87e9f3a5b596" containerName="revoke-keys"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139049    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139055    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="5eacbbd1-2f77-4095-aac9-6387699aa6f1" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139103    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139111    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:45:00.139157 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.139117    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="20b96c6f-5f11-411d-837e-87e9f3a5b596" containerName="revoke-keys"
Sep 16 14:45:00.141996 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.141977    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"
Sep 16 14:45:00.144165 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.144141    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-hkvhf\""
Sep 16 14:45:00.149644 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.149419    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"]
Sep 16 14:45:00.152685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.152662    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"
Sep 16 14:45:00.424621 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.424593    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"]
Sep 16 14:45:00.427210 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:45:00.427183    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cc9498b_1f1c_49e2_a950_493fdac3ac9a.slice/crio-8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770 WatchSource:0}: Error finding container 8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770: Status 404 returned error can't find the container with id 8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770
Sep 16 14:45:00.491864 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:00.491830    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp" event={"ID":"8cc9498b-1f1c-49e2-a950-493fdac3ac9a","Type":"ContainerStarted","Data":"8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770"}
Sep 16 14:45:01.498180 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:01.498147    2580 generic.go:356] "Generic (PLEG): container finished" podID="8cc9498b-1f1c-49e2-a950-493fdac3ac9a" containerID="1595bf53f4a351d44a883c2d87d5fb171f2ab654c1b33c56b43c6ffa3d390132" exitCode=0
Sep 16 14:45:01.498588 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:01.498210    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp" event={"ID":"8cc9498b-1f1c-49e2-a950-493fdac3ac9a","Type":"ContainerDied","Data":"1595bf53f4a351d44a883c2d87d5fb171f2ab654c1b33c56b43c6ffa3d390132"}
Sep 16 14:45:02.631339 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:02.631313    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"
Sep 16 14:45:03.508764 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:03.508682    2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp"
Sep 16 14:45:03.508764 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:03.508711    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29826165-q7gbp" event={"ID":"8cc9498b-1f1c-49e2-a950-493fdac3ac9a","Type":"ContainerDied","Data":"8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770"}
Sep 16 14:45:03.508764 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:03.508757    2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8df92b2b8c25dcb3c7802301af3a7d5ddc7b101326f7c0baef9466b3410fb770"
Sep 16 14:45:17.179000 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:45:17.178963    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:46:25.179117 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:46:25.179078    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:46:59.526974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:46:59.526903    2580 scope.go:117] "RemoveContainer" containerID="7f3e5d04654f837a03259dede5ea3e46014d58060cb086dfdf5ef19859a955f8"
Sep 16 14:46:59.534850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:46:59.534828    2580 scope.go:117] "RemoveContainer" containerID="f963f8f401117577a1714235f16540c0636e20c1a26dbdf589879dc05636a8b8"
Sep 16 14:47:29.181951 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:47:29.181918    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:47:59.555328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:47:59.555299    2580 scope.go:117] "RemoveContainer" containerID="00dc1de4164d5110fa8fcaae4354f409dd10a6dd50723ed54391681eb018549a"
Sep 16 14:48:53.178676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:48:53.178637    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:48:59.574515 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:48:59.574480    2580 scope.go:117] "RemoveContainer" containerID="55979a0c6cb45fb5b7dee73fbd1f96fd6fda99162e48b219d98974e410bf5fc0"
Sep 16 14:48:59.582630 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:48:59.582601    2580 scope.go:117] "RemoveContainer" containerID="2cb01d45dd3ae3d03d900a86fab8c07d5584e180e346408e18ef84e498728ae8"
Sep 16 14:50:08.178148 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:50:08.178115    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:51:31.178198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:51:31.178163    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:52:36.178752 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:36.178725    2580 kubelet_pods.go:1085] "Unable to retrieve pull secret, the image pull may not succeed." pod="openshift-ingress/payload-processing-5d7dbbd56b-n28jg" secret="" err="secret \"payload-processing-dockercfg-mzsgb\" not found"
Sep 16 14:52:37.096964 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:37.096935    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-klv79_ddcf2bf3-dad0-47a2-a1b1-a852918c4591/global-pull-secret-syncer/0.log"
Sep 16 14:52:37.265190 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:37.265165    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-zgc2b_3029ea23-b949-46e9-8b79-03ffcbf38066/konnectivity-agent/0.log"
Sep 16 14:52:37.324547 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:37.324518    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-80.ec2.internal_2a3ee3830b26f7f8f7ff158b9a7f8f57/haproxy/0.log"
Sep 16 14:52:43.824379 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.824347    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/alertmanager/0.log"
Sep 16 14:52:43.841149 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.841124    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/config-reloader/0.log"
Sep 16 14:52:43.855723 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.855697    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/kube-rbac-proxy-web/0.log"
Sep 16 14:52:43.874087 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.874067    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/kube-rbac-proxy/0.log"
Sep 16 14:52:43.889324 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.889302    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/kube-rbac-proxy-metric/0.log"
Sep 16 14:52:43.908333 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.908317    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/prom-label-proxy/0.log"
Sep 16 14:52:43.925328 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:43.925309    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_5ba33eff-0102-47bd-9fa7-06fc76a6df17/init-config-reloader/0.log"
Sep 16 14:52:44.052990 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.052968    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6488dffc95-pksr7_5b39ebf5-2a88-49a0-a0f7-833344564b2f/metrics-server/0.log"
Sep 16 14:52:44.105517 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.105492    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-588tg_4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b/node-exporter/0.log"
Sep 16 14:52:44.122212 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.122192    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-588tg_4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b/kube-rbac-proxy/0.log"
Sep 16 14:52:44.138183 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.138166    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-588tg_4128f4e8-c6fa-4e48-9dc1-cbb2a05c6b3b/init-textfile/0.log"
Sep 16 14:52:44.385099 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.385069    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/prometheus/0.log"
Sep 16 14:52:44.400883 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.400861    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/config-reloader/0.log"
Sep 16 14:52:44.421603 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.421582    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/thanos-sidecar/0.log"
Sep 16 14:52:44.451712 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.451694    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/kube-rbac-proxy-web/0.log"
Sep 16 14:52:44.474421 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.474401    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/kube-rbac-proxy/0.log"
Sep 16 14:52:44.497810 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.497780    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/kube-rbac-proxy-thanos/0.log"
Sep 16 14:52:44.520225 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.520206    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_b0fed74a-cb53-4aef-befb-7bc5d2cd013d/init-config-reloader/0.log"
Sep 16 14:52:44.600550 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.600527    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-5864cd99d7-vqqhc_d0e94b69-6556-4ab6-8f7f-2e3295b5e525/prometheus-operator-admission-webhook/0.log"
Sep 16 14:52:44.630850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.630789    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6758f99d-lvm6q_cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909/telemeter-client/0.log"
Sep 16 14:52:44.650139 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.650117    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6758f99d-lvm6q_cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909/reload/0.log"
Sep 16 14:52:44.672243 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:44.672222    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7f6758f99d-lvm6q_cbdbf2f6-2ce5-45e2-a152-6d6c5ea8e909/kube-rbac-proxy/0.log"
Sep 16 14:52:47.412941 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.412910    2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"]
Sep 16 14:52:47.413420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.413240    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8cc9498b-1f1c-49e2-a950-493fdac3ac9a" containerName="cleanup"
Sep 16 14:52:47.413420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.413251    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="8cc9498b-1f1c-49e2-a950-493fdac3ac9a" containerName="cleanup"
Sep 16 14:52:47.413420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.413264    2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:52:47.413420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.413270    2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5b6052c-f79f-4707-8dda-60704d2bd6a6" containerName="istio-proxy"
Sep 16 14:52:47.413420 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.413334    2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="8cc9498b-1f1c-49e2-a950-493fdac3ac9a" containerName="cleanup"
Sep 16 14:52:47.416230 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.416213    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.418292 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.418267    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8b222\"/\"openshift-service-ca.crt\""
Sep 16 14:52:47.418436 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.418420    2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-8b222\"/\"kube-root-ca.crt\""
Sep 16 14:52:47.418973 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.418957    2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-8b222\"/\"default-dockercfg-bwccg\""
Sep 16 14:52:47.422990 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.422965    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"]
Sep 16 14:52:47.570774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.570739    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-proc\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.570774 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.570774    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-lib-modules\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.570974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.570821    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xbzf4\" (UniqueName: \"kubernetes.io/projected/cfc159a3-67cc-4db9-8ac2-10652819a88c-kube-api-access-xbzf4\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.570974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.570858    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-podres\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.570974 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.570876    2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-sys\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.671974    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-proc\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672005    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-lib-modules\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672051 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672038    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xbzf4\" (UniqueName: \"kubernetes.io/projected/cfc159a3-67cc-4db9-8ac2-10652819a88c-kube-api-access-xbzf4\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672061    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-podres\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672077    2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-sys\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672104    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-proc\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672154    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-lib-modules\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672159    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-sys\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.672307 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.672213    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cfc159a3-67cc-4db9-8ac2-10652819a88c-podres\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.679901 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.679881    2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xbzf4\" (UniqueName: \"kubernetes.io/projected/cfc159a3-67cc-4db9-8ac2-10652819a88c-kube-api-access-xbzf4\") pod \"perf-node-gather-daemonset-xbg6t\" (UID: \"cfc159a3-67cc-4db9-8ac2-10652819a88c\") " pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.728983 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.728952    2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:47.998729 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:47.998699    2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"]
Sep 16 14:52:48.001928 ip-10-0-138-80 kubenswrapper[2580]: W0916 14:52:48.001895    2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcfc159a3_67cc_4db9_8ac2_10652819a88c.slice/crio-17da4d689d6669c01023488bc4ce0a1b900abb390e0ddb07fc9eafdccffc3cd8 WatchSource:0}: Error finding container 17da4d689d6669c01023488bc4ce0a1b900abb390e0ddb07fc9eafdccffc3cd8: Status 404 returned error can't find the container with id 17da4d689d6669c01023488bc4ce0a1b900abb390e0ddb07fc9eafdccffc3cd8
Sep 16 14:52:48.675807 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:48.675770    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t" event={"ID":"cfc159a3-67cc-4db9-8ac2-10652819a88c","Type":"ContainerStarted","Data":"3d84fc4fd59c0cc4395644548a91610634d197812cddd8a2269bc3a0afbc548b"}
Sep 16 14:52:48.676188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:48.675826    2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t" event={"ID":"cfc159a3-67cc-4db9-8ac2-10652819a88c","Type":"ContainerStarted","Data":"17da4d689d6669c01023488bc4ce0a1b900abb390e0ddb07fc9eafdccffc3cd8"}
Sep 16 14:52:48.676188 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:48.675877    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:48.691456 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:48.691412    2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t" podStartSLOduration=1.691399358 podStartE2EDuration="1.691399358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-16 14:52:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-16 14:52:48.689881631 +0000 UTC m=+2690.096461851" watchObservedRunningTime="2026-09-16 14:52:48.691399358 +0000 UTC m=+2690.097979565"
Sep 16 14:52:49.313850 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:49.313825    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-njkrv_aeebefad-ae12-4cb5-9cbf-516972fe339e/dns/0.log"
Sep 16 14:52:49.328645 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:49.328616    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-njkrv_aeebefad-ae12-4cb5-9cbf-516972fe339e/kube-rbac-proxy/0.log"
Sep 16 14:52:49.404837 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:49.404813    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-kz8kh_a1c0520e-773d-4d35-8f4a-394f677fea8d/dns-node-resolver/0.log"
Sep 16 14:52:50.050281 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:50.050247    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-lgpt4_17ba2d77-eb74-4d56-a93f-7df4d41b2ac5/node-ca/0.log"
Sep 16 14:52:50.977430 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:50.977403    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cktcsl_5735668c-474c-42f8-9080-8169919886e0/istio-proxy/0.log"
Sep 16 14:52:51.608349 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:51.608311    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6r7r_c9ba357f-a0e0-4c31-8480-c2d35f27e564/discovery/0.log"
Sep 16 14:52:51.671389 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:51.671361    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/0.log"
Sep 16 14:52:51.675610 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:51.675588    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/1.log"
Sep 16 14:52:51.712293 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:51.712271    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:52:52.588767 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:52.588742    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-97pq7_9aacf9cf-706a-401c-b3b1-73f57310a096/serve-healthcheck-canary/0.log"
Sep 16 14:52:53.093552 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:53.093525    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2plzs_0479fe57-ca07-4f5c-9059-981c5ffa9f79/kube-rbac-proxy/0.log"
Sep 16 14:52:53.110544 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:53.110525    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2plzs_0479fe57-ca07-4f5c-9059-981c5ffa9f79/exporter/0.log"
Sep 16 14:52:53.126676 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:53.126657    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-2plzs_0479fe57-ca07-4f5c-9059-981c5ffa9f79/extractor/0.log"
Sep 16 14:52:54.689986 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:54.689955    2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-8b222/perf-node-gather-daemonset-xbg6t"
Sep 16 14:52:55.287344 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:55.287300    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_ai-gateway-controller-d8854dcc5-6kvrk_56ede454-afb2-4be2-96bd-e154a6c37fd3/manager/0.log"
Sep 16 14:52:55.948446 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:55.948420    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-cv5gw_dce8ca82-336f-4cc2-9a2e-06b058199aab/manager/0.log"
Sep 16 14:52:56.140441 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:56.140414    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d8b99777-w22dd_e9940bca-6175-4bcb-9a3b-874acf183af3/manager/0.log"
Sep 16 14:52:56.157882 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:56.157856    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-58d8b99777-w22dd_e9940bca-6175-4bcb-9a3b-874acf183af3/copy-manifests/0.log"
Sep 16 14:52:57.679084 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:57.678996    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-8544d9d757-nm5t2_ce6111d7-479b-4c25-b1ff-c736d3a4c766/manager/0.log"
Sep 16 14:52:57.700055 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:57.700033    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-wmvfp_4b097f85-16ba-49f7-acda-a046bee40de0/openshift-lws-operator/0.log"
Sep 16 14:52:57.889121 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:57.889092    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cktcsl_5735668c-474c-42f8-9080-8169919886e0/istio-proxy/0.log"
Sep 16 14:52:58.516329 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:58.516298    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6r7r_c9ba357f-a0e0-4c31-8480-c2d35f27e564/discovery/0.log"
Sep 16 14:52:58.561138 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:58.561099    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/0.log"
Sep 16 14:52:58.565459 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:58.565433    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/1.log"
Sep 16 14:52:58.585491 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:52:58.585461    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:53:00.086727 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:00.086693    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-68cf4d799cktcsl_5735668c-474c-42f8-9080-8169919886e0/istio-proxy/0.log"
Sep 16 14:53:00.740982 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:00.740951    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-5677ccb49b-q6r7r_c9ba357f-a0e0-4c31-8480-c2d35f27e564/discovery/0.log"
Sep 16 14:53:00.793462 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:00.793426    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/0.log"
Sep 16 14:53:00.797888 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:00.797866    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-5ccb968c66-zf46t_426402ad-507c-4227-96a5-70718b421c25/istio-proxy/1.log"
Sep 16 14:53:00.821280 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:00.821260    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5d7dbbd56b-n28jg_0fa89d75-5a6e-461f-8be3-3d5018ae82dc/payload-processing/0.log"
Sep 16 14:53:04.098198 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.098170    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/kube-multus-additional-cni-plugins/0.log"
Sep 16 14:53:04.112763 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.112742    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/egress-router-binary-copy/0.log"
Sep 16 14:53:04.127372 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.127342    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/cni-plugins/0.log"
Sep 16 14:53:04.141924 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.141902    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/bond-cni-plugin/0.log"
Sep 16 14:53:04.157332 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.157307    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/routeoverride-cni/0.log"
Sep 16 14:53:04.173480 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.173448    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/whereabouts-cni-bincopy/0.log"
Sep 16 14:53:04.189835 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.189816    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mh2p9_bee02a73-e799-45f1-88c1-91c30f3428c0/whereabouts-cni/0.log"
Sep 16 14:53:04.434968 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.434886    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mvj4t_53838c2b-0d4c-4313-89f4-28728b38e7c2/kube-multus/0.log"
Sep 16 14:53:04.582021 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.581996    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xtwhx_85aa1096-f9c0-4677-a9a4-e86c23d4c4ea/network-metrics-daemon/0.log"
Sep 16 14:53:04.596685 ip-10-0-138-80 kubenswrapper[2580]: I0916 14:53:04.596665    2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-xtwhx_85aa1096-f9c0-4677-a9a4-e86c23d4c4ea/kube-rbac-proxy/0.log"