Sep 18 19:59:02.791772 ip-10-0-131-20 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 18 19:59:02.791784 ip-10-0-131-20 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 18 19:59:02.791794 ip-10-0-131-20 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 18 19:59:02.792122 ip-10-0-131-20 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 18 19:59:12.965996 ip-10-0-131-20 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 18 19:59:12.966010 ip-10-0-131-20 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 18718f37284941798bdf04c39c78d533 --
Sep 18 20:00:54.409343 ip-10-0-131-20 systemd[1]: Starting Kubernetes Kubelet...
Sep 18 20:00:57.884213 ip-10-0-131-20 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:00:57.884213 ip-10-0-131-20 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 18 20:00:57.884213 ip-10-0-131-20 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:00:57.884213 ip-10-0-131-20 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 18 20:00:57.884213 ip-10-0-131-20 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 18 20:00:57.885170 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.884282    2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890696    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890713    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890717    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890720    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890723    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.890718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890729    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890733    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890736    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890740    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890744    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890747    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890750    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890753    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890756    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890759    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890762    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890764    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890767    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890770    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890772    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890775    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890777    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890780    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890783    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890785    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.890935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890788    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890790    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890793    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890795    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890798    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890801    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890804    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890807    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890809    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890813    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890817    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890819    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890822    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890825    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890828    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890831    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890834    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890836    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890839    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890841    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.891433 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890844    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890847    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890849    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890852    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890855    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890857    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890860    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890862    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890865    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890868    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890870    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890873    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890875    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890878    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890880    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890883    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890885    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890890    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890894    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.891916 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890897    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890900    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890903    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890906    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890909    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890912    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890915    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890918    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890920    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890923    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890925    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890928    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890931    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890933    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890936    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890939    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890941    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890944    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890946    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890956    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.892391 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890959    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890961    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890964    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890967    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890969    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890972    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890975    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890979    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890983    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890985    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890988    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890990    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890993    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890995    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.890998    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891745    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891753    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891757    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891760    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891764    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.892877 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891767    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891770    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891773    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891776    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891778    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891782    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891784    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891787    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891790    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891792    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891795    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891798    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891806    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891809    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891812    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891814    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891817    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891820    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891822    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891825    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.893368 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891827    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891830    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891832    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891835    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891838    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891840    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891843    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891846    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891848    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891853    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891856    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891859    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891861    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891864    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891866    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891869    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891871    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891874    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891877    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.893902 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891880    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891883    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891885    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891888    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891890    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891893    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891901    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891904    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891909    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891912    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891915    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891918    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891922    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891924    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891927    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891930    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891934    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891937    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.894405 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891943    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891946    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891949    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891952    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891955    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891957    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891961    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891964    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891966    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891969    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891972    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891974    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891977    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891980    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891982    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891985    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891987    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891990    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891992    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891995    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.891998    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.894851 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892001    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892004    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892007    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892010    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892012    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892015    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892017    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892020    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892022    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892025    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892027    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892031    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892034    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892036    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892039    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892041    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892448    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892455    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892458    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892461    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.895389 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892464    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892467    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892470    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892472    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892475    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892478    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892480    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892483    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892485    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892488    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892490    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892493    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892496    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892501    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892504    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892506    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892509    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892511    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892514    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892517    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.895893 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892520    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892522    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892525    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892528    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892530    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892533    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892535    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892538    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892540    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892543    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892545    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892547    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892550    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892552    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892555    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892557    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892560    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892562    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892565    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892567    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.896383 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892570    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892572    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892574    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892577    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892579    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892584    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892588    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892591    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892594    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892597    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892600    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892603    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892605    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892608    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892611    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892613    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892616    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892619    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892621    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.896865 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892624    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892626    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892629    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892632    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892635    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892637    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892639    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892642    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892644    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892647    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892649    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892652    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892654    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892657    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892659    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892662    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892665    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892668    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892672    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892675    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.897373 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892678    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892680    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892682    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892685    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892687    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892690    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892692    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892697    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892700    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892703    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892706    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892712    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892715    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892717    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892720    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.892722    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892786    2566 flags.go:64] FLAG: --address="0.0.0.0"
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892819    2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892826    2566 flags.go:64] FLAG: --anonymous-auth="true"
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892830    2566 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892834    2566 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 18 20:00:57.897863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892838    2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892842    2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892846    2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892849    2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892852    2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892856    2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892859    2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892862    2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892866    2566 flags.go:64] FLAG: --cgroup-root=""
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892870    2566 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892873    2566 flags.go:64] FLAG: --client-ca-file=""
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892876    2566 flags.go:64] FLAG: --cloud-provider="external"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892880    2566 flags.go:64] FLAG: --cluster-dns="[]"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892884    2566 flags.go:64] FLAG: --cluster-domain=""
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892887    2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892890    2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892893    2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892897    2566 flags.go:64] FLAG: --container-log-max-files="5"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892901    2566 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892904    2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892907    2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892910    2566 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892913    2566 flags.go:64] FLAG: --contention-profiling="false"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892916    2566 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 18 20:00:57.898388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892920    2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892923    2566 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892926    2566 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892930    2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892933    2566 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892936    2566 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892939    2566 flags.go:64] FLAG: --enable-load-reader="false"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892942    2566 flags.go:64] FLAG: --enable-server="true"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892945    2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892949    2566 flags.go:64] FLAG: --event-burst="100"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892953    2566 flags.go:64] FLAG: --event-qps="50"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892956    2566 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892958    2566 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892961    2566 flags.go:64] FLAG: --eviction-hard=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892965    2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892968    2566 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892971    2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892975    2566 flags.go:64] FLAG: --eviction-soft=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892978    2566 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892981    2566 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892984    2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892987    2566 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892992    2566 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892995    2566 flags.go:64] FLAG: --fail-swap-on="true"
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.892998    2566 flags.go:64] FLAG: --feature-gates=""
Sep 18 20:00:57.899051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893002    2566 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893005    2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893008    2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893011    2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893014    2566 flags.go:64] FLAG: --healthz-port="10248"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893017    2566 flags.go:64] FLAG: --help="false"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893020    2566 flags.go:64] FLAG: --hostname-override="ip-10-0-131-20.ec2.internal"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893023    2566 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893026    2566 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893029    2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893033    2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893036    2566 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893039    2566 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893042    2566 flags.go:64] FLAG: --image-service-endpoint=""
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893045    2566 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893048    2566 flags.go:64] FLAG: --kube-api-burst="100"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893050    2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893068    2566 flags.go:64] FLAG: --kube-api-qps="50"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893072    2566 flags.go:64] FLAG: --kube-reserved=""
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893075    2566 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893078    2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893081    2566 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893084    2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893087    2566 flags.go:64] FLAG: --lock-file=""
Sep 18 20:00:57.899698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893091    2566 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893094    2566 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893097    2566 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893103    2566 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893110    2566 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893115    2566 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893118    2566 flags.go:64] FLAG: --logging-format="text"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893121    2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893124    2566 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893127    2566 flags.go:64] FLAG: --manifest-url=""
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893130    2566 flags.go:64] FLAG: --manifest-url-header=""
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893134    2566 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893137    2566 flags.go:64] FLAG: --max-open-files="1000000"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893141    2566 flags.go:64] FLAG: --max-pods="110"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893144    2566 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893147    2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893150    2566 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893153    2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893156    2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893159    2566 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893162    2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893170    2566 flags.go:64] FLAG: --node-status-max-images="50"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893173    2566 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893176    2566 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 18 20:00:57.900340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893179    2566 flags.go:64] FLAG: --pod-cidr=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893181    2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:a6abe77af61c5f2e3985a83e7d43538aefc38b1b0444127da14ee0e8cf8f76fc"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893186    2566 flags.go:64] FLAG: --pod-manifest-path=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893189    2566 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893192    2566 flags.go:64] FLAG: --pods-per-core="0"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893195    2566 flags.go:64] FLAG: --port="10250"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893198    2566 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893201    2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0c3a953216c240675"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893205    2566 flags.go:64] FLAG: --qos-reserved=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893208    2566 flags.go:64] FLAG: --read-only-port="10255"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893211    2566 flags.go:64] FLAG: --register-node="true"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893214    2566 flags.go:64] FLAG: --register-with-taints=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893217    2566 flags.go:64] FLAG: --registry-burst="10"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893221    2566 flags.go:64] FLAG: --registry-qps="5"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893225    2566 flags.go:64] FLAG: --reserved-cpus=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893228    2566 flags.go:64] FLAG: --reserved-memory=""
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893232    2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893235    2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893238    2566 flags.go:64] FLAG: --rotate-certificates="false"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893241    2566 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893244    2566 flags.go:64] FLAG: --runonce="false"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893247    2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893250    2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893253    2566 flags.go:64] FLAG: --seccomp-default="false"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893257    2566 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 18 20:00:57.900939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893259    2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893262    2566 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893265    2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893268    2566 flags.go:64] FLAG: --storage-driver-password="root"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893271    2566 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893274    2566 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893277    2566 flags.go:64] FLAG: --storage-driver-user="root"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893280    2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893282    2566 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893285    2566 flags.go:64] FLAG: --system-cgroups=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893288    2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893293    2566 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893296    2566 flags.go:64] FLAG: --tls-cert-file=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893299    2566 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893303    2566 flags.go:64] FLAG: --tls-min-version=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893307    2566 flags.go:64] FLAG: --tls-private-key-file=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893310    2566 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893312    2566 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893315    2566 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893318    2566 flags.go:64] FLAG: --v="2"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893322    2566 flags.go:64] FLAG: --version="false"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893328    2566 flags.go:64] FLAG: --vmodule=""
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893332    2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.893335    2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893445    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.901563 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893449    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893452    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893455    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893457    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893460    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893463    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893465    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893468    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893471    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893473    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893476    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893478    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893481    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893484    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893486    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893489    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893492    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893494    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893497    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893500    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.902202 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893502    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893505    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893511    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893514    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893517    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893520    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893523    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893526    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893531    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893535    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893538    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893541    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893544    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893548    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893551    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893554    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893556    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893559    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893562    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.902734 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893564    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893567    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893576    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893578    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893581    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893584    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893586    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893589    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893591    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893594    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893597    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893599    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893602    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893604    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893607    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893611    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893614    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893616    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893619    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.903305 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893622    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893624    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893628    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893631    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893635    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893638    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893641    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893643    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893646    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893650    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893654    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893657    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893661    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893664    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893669    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893673    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893675    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893678    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893680    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893683    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.903771 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893686    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893689    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893691    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893694    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893696    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893699    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893702    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893704    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893709    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893713    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893715    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893718    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893720    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893723    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893728    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893730    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893733    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893736    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893739    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.904262 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.893741    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.894544    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.901453    2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.901467    2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.901486    2566 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.901494    2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901552    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901557    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901560    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901564    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901567    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901570    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901573    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901576    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901579    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.904772 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901582    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901585    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901587    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901590    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901593    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901596    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901598    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901601    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901604    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901606    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901609    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901611    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901615    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901619    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901622    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901626    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901629    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901632    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901635    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.905145 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901637    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901641    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901644    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901648    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901652    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901655    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901658    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901660    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901663    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901666    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901669    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901672    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901674    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901677    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901680    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901683    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901685    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901688    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901692    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.905625 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901695    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901698    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901701    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901704    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901707    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901709    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901712    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901715    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901718    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901721    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901724    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901727    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901730    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901733    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901737    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901739    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901742    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901745    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901747    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901750    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.906162 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901753    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901755    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901758    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901760    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901763    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901766    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901768    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901771    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901774    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901776    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901779    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901781    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901784    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901787    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901789    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901792    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901794    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901797    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901799    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901802    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.906653 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901805    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901808    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901810    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901813    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901816    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901819    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901822    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901825    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901827    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901830    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901833    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901835    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.901840    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901954    2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901958    2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 18 20:00:57.907158 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901961    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901964    2566 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901967    2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901969    2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901972    2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901975    2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901977    2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901981    2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901985    2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901988    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901991    2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901994    2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901996    2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.901999    2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902001    2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902004    2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902006    2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902009    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902012    2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 18 20:00:57.907527 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902014    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902017    2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902020    2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902023    2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902026    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902029    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902032    2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902034    2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902037    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902040    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902042    2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902045    2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902047    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902050    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902067    2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902072    2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902075    2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902077    2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902081    2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 18 20:00:57.908006 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902085    2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902088    2566 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902091    2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902094    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902097    2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902100    2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902103    2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902105    2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902108    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902111    2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902114    2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902116    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902119    2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902121    2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902124    2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902126    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902130    2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902133    2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902137    2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902141    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 18 20:00:57.908502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902143    2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902147    2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902149    2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902152    2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902155    2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902157    2566 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902160    2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902163    2566 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902165    2566 feature_gate.go:328] unrecognized feature gate: Example
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902168    2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902170    2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902173    2566 feature_gate.go:328] unrecognized feature gate: Example2
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902175    2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902178    2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902180    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902183    2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902185    2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902188    2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902191    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902193    2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 18 20:00:57.908987 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902196    2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902198    2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902201    2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902203    2566 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902206    2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902209    2566 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902212    2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902214    2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902217    2566 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902220    2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902223    2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902226    2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902228    2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902231    2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902233    2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902235    2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902238    2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902240    2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 18 20:00:57.909536 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:57.902243    2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.902248    2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.902334    2566 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.904352    2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.905257    2566 server.go:1019] "Starting client certificate rotation"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.905349    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.906196    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.907769    2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:00:57.909972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.909208    2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 18 20:00:57.915162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.915145    2566 log.go:25] "Validated CRI v1 runtime API"
Sep 18 20:00:57.916418 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.916406    2566 log.go:25] "Validated CRI v1 image API"
Sep 18 20:00:57.917102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.917089    2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 18 20:00:57.920127 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.920107    2566 fs.go:135] Filesystem UUIDs: map[52f14bfe-b0ef-45ed-aba4-9d8da06fc2be:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e013c0b6-1c27-484d-a223-2fe5d4dd340f:/dev/nvme0n1p4]
Sep 18 20:00:57.920187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.920125    2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 18 20:00:57.924196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.924085    2566 manager.go:216] Machine: {Timestamp:2026-09-18 20:00:57.92361711 +0000 UTC m=+2.052919051 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098466 MemoryCapacity:33164472320 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec230b2d1d0c754edb1fd308463a9808 SystemUUID:ec230b2d-1d0c-754e-db1f-d308463a9808 BootID:18718f37-2849-4179-8bdf-04c39c78d533 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632894464 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582234112 Type:vfs Inodes:4048397 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:ff:ee:84:71:6d Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:ff:ee:84:71:6d Speed:0 Mtu:9001} {Name:ovs-system MacAddress:2a:75:99:0a:aa:f5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164472320 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 18 20:00:57.924196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.924188    2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 18 20:00:57.924366 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.924257    2566 manager.go:232] Version: {KernelVersion:5.14.0-570.141.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260914-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 18 20:00:57.925980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.925959    2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 18 20:00:57.926118 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.925981    2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-131-20.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 18 20:00:57.927226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.927216    2566 topology_manager.go:138] "Creating topology manager with none policy"
Sep 18 20:00:57.927263 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.927229    2566 container_manager_linux.go:309] "Creating device plugin manager"
Sep 18 20:00:57.927263 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.927237    2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:00:57.927263 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.927247    2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 18 20:00:57.927345 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.927310    2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 18 20:00:57.931821 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931809    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:00:57.931930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931919    2566 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 18 20:00:57.931965 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931952    2566 kubelet.go:486] "Attempting to sync node with API server"
Sep 18 20:00:57.931965 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931961    2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 18 20:00:57.932027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931974    2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 18 20:00:57.932027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931982    2566 kubelet.go:398] "Adding apiserver pod source"
Sep 18 20:00:57.932027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.931991    2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 18 20:00:57.932283 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.932270    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:00:57.932328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.932296    2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 18 20:00:57.938241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.938219    2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.13-6.rhaos4.21.git6253162.el9" apiVersion="v1"
Sep 18 20:00:57.938335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.938311    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:00:57.940539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.940525    2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 18 20:00:57.940591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.940555    2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 18 20:00:57.941694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941682    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941703    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941710    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941716    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941725    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941731    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 18 20:00:57.941744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941744    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941751    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941774    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941780    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941801    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941806    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941828    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 18 20:00:57.941902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.941835    2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 18 20:00:57.944208 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.944187    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 18 20:00:57.944277 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.944246    2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-131-20.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 18 20:00:57.944313 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.944296    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-131-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 18 20:00:57.945823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.945809    2566 server.go:1268] "Started kubelet"
Sep 18 20:00:57.945891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.945871    2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 18 20:00:57.945945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.945886    2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 18 20:00:57.945979 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.945957    2566 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 18 20:00:57.969043 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.969003    2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 18 20:00:57.969696 ip-10-0-131-20 systemd[1]: Started Kubernetes Kubelet.
Sep 18 20:00:57.970431 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.970403    2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 18 20:00:57.970494 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.970440    2566 server.go:310] "Adding debug handlers to kubelet server"
Sep 18 20:00:57.975237 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.974276    2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-131-20.ec2.internal.18d682bece78f6d6  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-131-20.ec2.internal,UID:ip-10-0-131-20.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-131-20.ec2.internal,},FirstTimestamp:2026-09-18 20:00:57.945781974 +0000 UTC m=+2.075083916,LastTimestamp:2026-09-18 20:00:57.945781974 +0000 UTC m=+2.075083916,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-131-20.ec2.internal,}"
Sep 18 20:00:57.978130 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.978114    2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 18 20:00:57.978207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.978128    2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 18 20:00:57.978275 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.978260    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:57.978961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.978944    2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 18 20:00:57.978961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.978944    2566 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 18 20:00:57.979074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.978967    2566 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 18 20:00:57.979074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.979034    2566 reconstruct.go:97] "Volume reconstruction finished"
Sep 18 20:00:57.979074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.979045    2566 reconciler.go:29] "Reconciler: start to sync state"
Sep 18 20:00:57.979822 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.979807    2566 factory.go:55] Registering systemd factory
Sep 18 20:00:57.979822 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.979823    2566 factory.go:223] Registration of the systemd container factory successfully
Sep 18 20:00:57.980068 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980044    2566 factory.go:153] Registering CRI-O factory
Sep 18 20:00:57.980133 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980077    2566 factory.go:223] Registration of the crio container factory successfully
Sep 18 20:00:57.980133 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980112    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ftmfj"
Sep 18 20:00:57.980239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980144    2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 18 20:00:57.980239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980163    2566 factory.go:103] Registering Raw factory
Sep 18 20:00:57.980239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980176    2566 manager.go:1196] Started watching for new ooms in manager
Sep 18 20:00:57.980752 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.980732    2566 manager.go:349] Starting recovery of all containers
Sep 18 20:00:57.987562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.987538    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-ftmfj"
Sep 18 20:00:57.989786 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.989761    2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 18 20:00:57.989894 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:57.989804    2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-131-20.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 18 20:00:57.990990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.990976    2566 manager.go:354] Recovery completed
Sep 18 20:00:57.995032 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.995017    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:57.996615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.996515    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:57.996713 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.996632    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:57.996713 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.996648    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:57.997233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997219    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:57.997282 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997221    2566 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 18 20:00:57.997282 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997278    2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 18 20:00:57.997374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997293    2566 state_mem.go:36] "Initialized new in-memory state store"
Sep 18 20:00:57.997820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997805    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:57.997867 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997829    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:57.997867 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.997838    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:57.999408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.999394    2566 policy_none.go:49] "None policy: Start"
Sep 18 20:00:57.999479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.999413    2566 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 18 20:00:57.999479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:57.999425    2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 18 20:00:58.001433 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.001420    2566 policy_none.go:47] "Start"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.048619    2566 manager.go:337] "Starting Device Plugin manager"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.048656    2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.048667    2566 server.go:87] "Starting device plugin registration server"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.048896    2566 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.048916    2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.049129    2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.049218    2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.049227    2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.049648    2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 18 20:00:58.050692 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.049700    2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.072595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.072571    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 18 20:00:58.073716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.073693    2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 18 20:00:58.073805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.073741    2566 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 18 20:00:58.073805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.073763    2566 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 18 20:00:58.073805 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.073800    2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 18 20:00:58.075755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.075735    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:00:58.149217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.149161    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:58.150443 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.150428    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:58.150541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.150459    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:58.150541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.150474    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:58.150541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.150503    2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.160379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.160362    2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.160446 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.160384    2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-131-20.ec2.internal\": node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.174610 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.174590    2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"]
Sep 18 20:00:58.174699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.174651    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:58.175414 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.175400    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:58.175493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.175428    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:58.175493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.175443    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:58.177631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.177617    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:58.177767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.177752    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.177812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.177781    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:58.178377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178363    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:58.178445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178384    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:58.178445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178368    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:58.178445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178396    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:58.178445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178415    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:58.178445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.178427    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:58.180290 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.180273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.180360 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.180301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.180408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.180342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/857cbacc60cccfe8f3eccb86ceb6ba23-config\") pod \"kube-apiserver-proxy-ip-10-0-131-20.ec2.internal\" (UID: \"857cbacc60cccfe8f3eccb86ceb6ba23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.180526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.180513    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.180568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.180536    2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 18 20:00:58.181368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.181344    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientMemory"
Sep 18 20:00:58.181447 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.181380    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasNoDiskPressure"
Sep 18 20:00:58.181447 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.181396    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeHasSufficientPID"
Sep 18 20:00:58.183315 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.183296    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.194655 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.194636    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-20.ec2.internal\" not found" node="ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.198552 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.198532    2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-131-20.ec2.internal\" not found" node="ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.280949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.280925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/857cbacc60cccfe8f3eccb86ceb6ba23-config\") pod \"kube-apiserver-proxy-ip-10-0-131-20.ec2.internal\" (UID: \"857cbacc60cccfe8f3eccb86ceb6ba23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.280949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.280951    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.281125 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.280966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.281125 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.281011    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.281125 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.281027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/857cbacc60cccfe8f3eccb86ceb6ba23-config\") pod \"kube-apiserver-proxy-ip-10-0-131-20.ec2.internal\" (UID: \"857cbacc60cccfe8f3eccb86ceb6ba23\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.281125 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.281040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/bc3eefcfaadb741284b07feb390941dd-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal\" (UID: \"bc3eefcfaadb741284b07feb390941dd\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.284050 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.284026    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.384501 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.384483    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.485318 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.485291    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.498441 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.498423    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.502154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.502132    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:58.585575 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.585546    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.686136 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.686107    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.786621 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.786549    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.887086 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.887051    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.905208 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.905188    2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 18 20:00:58.905321 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.905305    2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 18 20:00:58.988165 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:58.988139    2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-131-20.ec2.internal\" not found"
Sep 18 20:00:58.994274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.994245    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-17 19:55:57 +0000 UTC" deadline="2028-05-08 12:44:01.017877122 +0000 UTC"
Sep 18 20:00:58.994274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:58.994271    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14344h43m2.023608827s"
Sep 18 20:00:59.032014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.031991    2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:00:59.079701 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.079682    2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:00:59.100368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.100351    2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:00:59.178840 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.178804    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:59.192622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.192601    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:00:59.193478 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.193465    2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal"
Sep 18 20:00:59.199650 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.199638    2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 18 20:00:59.247101 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:59.247073    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod857cbacc60cccfe8f3eccb86ceb6ba23.slice/crio-a3a080411a2d8e945cc1b037257564bdf285e9e015509d1a2939dad76c684a39 WatchSource:0}: Error finding container a3a080411a2d8e945cc1b037257564bdf285e9e015509d1a2939dad76c684a39: Status 404 returned error can't find the container with id a3a080411a2d8e945cc1b037257564bdf285e9e015509d1a2939dad76c684a39
Sep 18 20:00:59.248539 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:00:59.247911    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc3eefcfaadb741284b07feb390941dd.slice/crio-f7f2ad28cb6fbe14cad9162439948e5928e4a8fa5cca08950d8edb3fd04ad244 WatchSource:0}: Error finding container f7f2ad28cb6fbe14cad9162439948e5928e4a8fa5cca08950d8edb3fd04ad244: Status 404 returned error can't find the container with id f7f2ad28cb6fbe14cad9162439948e5928e4a8fa5cca08950d8edb3fd04ad244
Sep 18 20:00:59.255297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.255278    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:00:59.932865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.932834    2566 apiserver.go:52] "Watching apiserver"
Sep 18 20:00:59.941250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.941224    2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 18 20:00:59.941733 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.941713    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-zbm8h","kube-system/konnectivity-agent-mvnvn","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd","openshift-cluster-node-tuning-operator/tuned-5dsjh","openshift-dns/node-resolver-67q5n","openshift-multus/multus-79qgh","openshift-multus/network-metrics-daemon-4jr7x","openshift-network-operator/iptables-alerter-5bwtn","kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal","openshift-image-registry/node-ca-7h7pk","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal","openshift-multus/multus-additional-cni-plugins-7r7ct","openshift-network-diagnostics/network-check-target-k546r","openshift-ovn-kubernetes/ovnkube-node-5xrvx"]
Sep 18 20:00:59.944782 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.944759    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:00:59.944900 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:59.944836    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:00:59.947100 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.946928    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:00:59.947100 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:59.947003    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:00:59.949438 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.949219    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.951245 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.951224    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 18 20:00:59.951615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.951596    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vp97k\""
Sep 18 20:00:59.951898 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.951881    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.952100 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.952083    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.952664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.952508    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.954349 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.953870    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:00:59.955105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.954705    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.955105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.954938    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-vdtd8\""
Sep 18 20:00:59.955822 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.955435    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.956287 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.955975    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.956387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.956346    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-c52tj\""
Sep 18 20:00:59.956813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.956705    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.958317 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.956933    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.959705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.959637    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.960640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.959836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-czwrm\""
Sep 18 20:00:59.960640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.960028    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 18 20:00:59.960640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.960200    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.960640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.960201    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 18 20:00:59.960640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.960444    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.962048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.962028    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:00:59.962155 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:00:59.962112    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:00:59.964000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.962584    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.964000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.963158    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-wmmll\""
Sep 18 20:00:59.964000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.963447    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 18 20:00:59.964000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.963928    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 18 20:00:59.964978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.964556    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.964978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.964579    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 18 20:00:59.964978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.964719    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:00:59.968118 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.967532    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:00:59.968916 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.968496    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 18 20:00:59.969317 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.969182    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 18 20:00:59.970183 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.970043    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 18 20:00:59.970247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.970048    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.972706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.970551    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:00:59.972706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.971111    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.972706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.971321    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f2jnd\""
Sep 18 20:00:59.972706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.971407    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 18 20:00:59.972706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.971589    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pbs5l\""
Sep 18 20:00:59.973041 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.972832    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 18 20:00:59.974084 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.974049    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nkq5k\""
Sep 18 20:00:59.975155 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.974982    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:00:59.975155 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.975096    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 18 20:00:59.975991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.975971    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.978071 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.978041    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 18 20:00:59.978393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.978187    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-kfm8w\""
Sep 18 20:00:59.978393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.978206    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 18 20:00:59.979101 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.979087    2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 18 20:00:59.980113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.980095    2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 18 20:00:59.987546 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987524    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-socket-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.987636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987557    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95phv\" (UniqueName: \"kubernetes.io/projected/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-kube-api-access-95phv\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:00:59.987636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987582    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.987636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987604    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-tuned\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.987636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-systemd-units\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-netns\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987683    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e10a48b8-18a8-415e-8240-5072512f4f1f-ovn-node-metrics-cert\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987705    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgskf\" (UniqueName: \"kubernetes.io/projected/e10a48b8-18a8-415e-8240-5072512f4f1f-kube-api-access-zgskf\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-serviceca\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987760    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sq6x9\" (UniqueName: \"kubernetes.io/projected/7edd8626-6905-4132-889b-d2f5043c018e-kube-api-access-sq6x9\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987809    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-multus\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.987832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987831    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-hostroot\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ntn8x\" (UniqueName: \"kubernetes.io/projected/e5266fe9-cd4f-4363-9255-b951f4a56d59-kube-api-access-ntn8x\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-log-socket\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987927    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-k8s-cni-cncf-io\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987950    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-kubelet\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987975    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85665792-b3e0-4b6a-b7f2-cab42e82cd62-agent-certs\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.987997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-registration-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988127    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25vqv\" (UniqueName: \"kubernetes.io/projected/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-kube-api-access-25vqv\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988186    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:00:59.988233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988217    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b69fb34b-9443-4cea-9b7c-19ba8258730a-hosts-file\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-system-cni-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988281    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-daemon-config\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988321    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-etc-kubernetes\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988345    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-sys-fs\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988417    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-modprobe-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988458    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-tmp\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-88xhn\" (UniqueName: \"kubernetes.io/projected/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-kube-api-access-88xhn\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-cnibin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988559    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-cni-binary-copy\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhmq9\" (UniqueName: \"kubernetes.io/projected/b69fb34b-9443-4cea-9b7c-19ba8258730a-kube-api-access-jhmq9\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988623    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-systemd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.988763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988744    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-etc-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-script-lib\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-bin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-dbus\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988913    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-device-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-host\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988943    2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988960    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-conf\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.988981    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-systemd\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-sys\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-netd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989073    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-kubelet-config\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989107    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85665792-b3e0-4b6a-b7f2-cab42e82cd62-konnectivity-ca\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-kubelet\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-slash\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989224    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-ovn\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989256    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-config\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.989385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-conf-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989312    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-multus-certs\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989353    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7edd8626-6905-4132-889b-d2f5043c018e-host-slash\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989383    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-run\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989406    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-lib-modules\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989429    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-system-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989454    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989488    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysconfig\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989527    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-kubernetes\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989550    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-var-lib-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989596    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cnibin\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989618    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczw6\" (UniqueName: \"kubernetes.io/projected/d7a6ca69-b215-437a-9191-d46540f76181-kube-api-access-rczw6\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989641    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989675    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hx98p\" (UniqueName: \"kubernetes.io/projected/c367a973-1121-4c33-895f-e2b2f4e60b0f-kube-api-access-hx98p\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989701    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-host\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989730    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-bin\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989765    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-socket-dir-parent\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989800    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-var-lib-kubelet\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989824    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69fb34b-9443-4cea-9b7c-19ba8258730a-tmp-dir\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989848    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989885    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-env-overrides\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989923    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7edd8626-6905-4132-889b-d2f5043c018e-iptables-alerter-script\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989965    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.989999    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.990027    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-node-log\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.990051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:00:59.990967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:00:59.990090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-netns\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.008356 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.008314    2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 18 20:01:00.010774 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.010754    2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x7pxm"
Sep 18 20:01:00.018963 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.018593    2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x7pxm"
Sep 18 20:01:00.080096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.080000    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal" event={"ID":"857cbacc60cccfe8f3eccb86ceb6ba23","Type":"ContainerStarted","Data":"a3a080411a2d8e945cc1b037257564bdf285e9e015509d1a2939dad76c684a39"}
Sep 18 20:01:00.082404 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.082372    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal" event={"ID":"bc3eefcfaadb741284b07feb390941dd","Type":"ContainerStarted","Data":"f7f2ad28cb6fbe14cad9162439948e5928e4a8fa5cca08950d8edb3fd04ad244"}
Sep 18 20:01:00.091183 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-systemd-units\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091193    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-netns\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e10a48b8-18a8-415e-8240-5072512f4f1f-ovn-node-metrics-cert\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091241    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgskf\" (UniqueName: \"kubernetes.io/projected/e10a48b8-18a8-415e-8240-5072512f4f1f-kube-api-access-zgskf\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091264    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-serviceca\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.091296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sq6x9\" (UniqueName: \"kubernetes.io/projected/7edd8626-6905-4132-889b-d2f5043c018e-kube-api-access-sq6x9\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091313    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091340    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-multus\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091361    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-hostroot\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091383    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ntn8x\" (UniqueName: \"kubernetes.io/projected/e5266fe9-cd4f-4363-9255-b951f4a56d59-kube-api-access-ntn8x\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091424    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-log-socket\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091443    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-k8s-cni-cncf-io\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091472    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-kubelet\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85665792-b3e0-4b6a-b7f2-cab42e82cd62-agent-certs\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091506    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091526    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-registration-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.091558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091548    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091571    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25vqv\" (UniqueName: \"kubernetes.io/projected/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-kube-api-access-25vqv\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b69fb34b-9443-4cea-9b7c-19ba8258730a-hosts-file\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091677    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-system-cni-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091715    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-log-socket\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091734    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-netns\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.092157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091746    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-registration-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.092652 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.092631    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-systemd-units\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.092767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.092653    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.092917 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.092898    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:00.093533 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.093196    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:00.593172975 +0000 UTC m=+4.722474926 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:00.093533 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.093402    2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 18 20:01:00.093730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.093603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/b69fb34b-9443-4cea-9b7c-19ba8258730a-hosts-file\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.093730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.093663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-multus\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.093955    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-serviceca\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.093978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-hostroot\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091714    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-daemon-config\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-etc-kubernetes\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094124    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-sys-fs\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-modprobe-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094223    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-tmp\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094257    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-88xhn\" (UniqueName: \"kubernetes.io/projected/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-kube-api-access-88xhn\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-cnibin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094449    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-cni-binary-copy\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhmq9\" (UniqueName: \"kubernetes.io/projected/b69fb34b-9443-4cea-9b7c-19ba8258730a-kube-api-access-jhmq9\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094522    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-systemd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-etc-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094562    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-system-cni-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094593    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-script-lib\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094596    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-daemon-config\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.095593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094663    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-k8s-cni-cncf-io\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.094798    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-etc-kubernetes\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095035    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-bin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-dbus\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095123    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-device-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-host\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095181    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-conf\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-systemd\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095230    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-sys\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095326    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095351    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-netd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095361    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-script-lib\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095382    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-kubelet-config\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095434    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85665792-b3e0-4b6a-b7f2-cab42e82cd62-konnectivity-ca\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095469    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-kubelet\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095472    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-etc-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095510    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-kubelet-config\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.096525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095516    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-slash\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095572    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-slash\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095633    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-kubelet\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095556    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-ovn\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095759    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-ovn\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095786    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-device-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095813    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-run-systemd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.095895    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.095965    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-modprobe-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096022    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-cni-bin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.096124    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:00.596098313 +0000 UTC m=+4.725400255 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096177    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/1abab961-0419-4eb6-b08f-78d713072033-dbus\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/85665792-b3e0-4b6a-b7f2-cab42e82cd62-konnectivity-ca\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-config\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096286    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-conf-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-cnibin\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096322    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-multus-certs\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7edd8626-6905-4132-889b-d2f5043c018e-host-slash\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.097324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096386    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-run\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096416    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-lib-modules\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096530    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-conf\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096576    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-system-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096617    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096645    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysconfig\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-kubernetes\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-var-lib-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096808    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-sys\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096737    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cnibin\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096880    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczw6\" (UniqueName: \"kubernetes.io/projected/d7a6ca69-b215-437a-9191-d46540f76181-kube-api-access-rczw6\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096893    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-netd\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hx98p\" (UniqueName: \"kubernetes.io/projected/c367a973-1121-4c33-895f-e2b2f4e60b0f-kube-api-access-hx98p\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.091764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-var-lib-kubelet\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097103    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-host\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.098270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097136    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-bin\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-socket-dir-parent\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097200    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-var-lib-kubelet\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69fb34b-9443-4cea-9b7c-19ba8258730a-tmp-dir\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097414    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097450    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-env-overrides\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097482    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7edd8626-6905-4132-889b-d2f5043c018e-iptables-alerter-script\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097507    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097539    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097681    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-node-log\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097743    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-netns\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-socket-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097789    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c367a973-1121-4c33-895f-e2b2f4e60b0f-cni-binary-copy\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097799    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-95phv\" (UniqueName: \"kubernetes.io/projected/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-kube-api-access-95phv\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097832    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097890    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-kubelet-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.099080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097939    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-system-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.097958    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-tuned\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098072    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-conf-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-host\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098137    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-multus-certs\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098183    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/7edd8626-6905-4132-889b-d2f5043c018e-host-slash\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098435    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-run\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098780    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-lib-modules\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-var-lib-openvswitch\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096619    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-systemd\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098904    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-ovnkube-config\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098911    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysconfig\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.098966    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-kubernetes\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099007    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-run-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099107    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cnibin\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099309    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-cni-bin\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099436    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/e10a48b8-18a8-415e-8240-5072512f4f1f-env-overrides\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.099882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.096946    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-sys-fs\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099500    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-host\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099523    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-var-lib-kubelet\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099605    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-socket-dir-parent\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099701    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-multus-cni-dir\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099754    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-socket-dir\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099759    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b69fb34b-9443-4cea-9b7c-19ba8258730a-tmp-dir\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099814    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c367a973-1121-4c33-895f-e2b2f4e60b0f-host-run-netns\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.099866    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100243    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/7edd8626-6905-4132-889b-d2f5043c018e-iptables-alerter-script\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100348    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100385    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/e10a48b8-18a8-415e-8240-5072512f4f1f-node-log\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100481    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-cni-binary-copy\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.100755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100499    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-sysctl-d\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.101379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.100996    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e5266fe9-cd4f-4363-9255-b951f4a56d59-etc-selinux\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.104828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.103773    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/e10a48b8-18a8-415e-8240-5072512f4f1f-ovn-node-metrics-cert\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.104828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.103841    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-tmp\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.104828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.104041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/85665792-b3e0-4b6a-b7f2-cab42e82cd62-agent-certs\") pod \"konnectivity-agent-mvnvn\" (UID: \"85665792-b3e0-4b6a-b7f2-cab42e82cd62\") " pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:00.105473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.105450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-etc-tuned\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.110168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.110125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ntn8x\" (UniqueName: \"kubernetes.io/projected/e5266fe9-cd4f-4363-9255-b951f4a56d59-kube-api-access-ntn8x\") pod \"aws-ebs-csi-driver-node-mlrbd\" (UID: \"e5266fe9-cd4f-4363-9255-b951f4a56d59\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.111272 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.111154    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:00.111272 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.111263    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:00.111424 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.111279    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:00.111424 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.111368    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:00.611349176 +0000 UTC m=+4.740651128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:00.111693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.111667    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-88xhn\" (UniqueName: \"kubernetes.io/projected/1f75758d-a29f-4ec6-b39e-73a4ed1a3c73-kube-api-access-88xhn\") pod \"tuned-5dsjh\" (UID: \"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73\") " pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.112240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.112194    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhmq9\" (UniqueName: \"kubernetes.io/projected/b69fb34b-9443-4cea-9b7c-19ba8258730a-kube-api-access-jhmq9\") pod \"node-resolver-67q5n\" (UID: \"b69fb34b-9443-4cea-9b7c-19ba8258730a\") " pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.112240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.112205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczw6\" (UniqueName: \"kubernetes.io/projected/d7a6ca69-b215-437a-9191-d46540f76181-kube-api-access-rczw6\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:00.112769 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.112609    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sq6x9\" (UniqueName: \"kubernetes.io/projected/7edd8626-6905-4132-889b-d2f5043c018e-kube-api-access-sq6x9\") pod \"iptables-alerter-5bwtn\" (UID: \"7edd8626-6905-4132-889b-d2f5043c018e\") " pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.112855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.112839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hx98p\" (UniqueName: \"kubernetes.io/projected/c367a973-1121-4c33-895f-e2b2f4e60b0f-kube-api-access-hx98p\") pod \"multus-79qgh\" (UID: \"c367a973-1121-4c33-895f-e2b2f4e60b0f\") " pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.113237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.112997    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25vqv\" (UniqueName: \"kubernetes.io/projected/e05ba3d9-56f2-4a3b-89b2-4511ace2afbf-kube-api-access-25vqv\") pod \"multus-additional-cni-plugins-7r7ct\" (UID: \"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf\") " pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.113388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.113345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgskf\" (UniqueName: \"kubernetes.io/projected/e10a48b8-18a8-415e-8240-5072512f4f1f-kube-api-access-zgskf\") pod \"ovnkube-node-5xrvx\" (UID: \"e10a48b8-18a8-415e-8240-5072512f4f1f\") " pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.114963 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.114934    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-95phv\" (UniqueName: \"kubernetes.io/projected/5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755-kube-api-access-95phv\") pod \"node-ca-7h7pk\" (UID: \"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755\") " pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.262409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.262380    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd"
Sep 18 20:01:00.275715 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:00.275377    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5266fe9_cd4f_4363_9255_b951f4a56d59.slice/crio-88ede0c1fb75b381e4a43f967e78a4c53591c48509cd6cf55131683b22a8c28a WatchSource:0}: Error finding container 88ede0c1fb75b381e4a43f967e78a4c53591c48509cd6cf55131683b22a8c28a: Status 404 returned error can't find the container with id 88ede0c1fb75b381e4a43f967e78a4c53591c48509cd6cf55131683b22a8c28a
Sep 18 20:01:00.275715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.275459    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5dsjh"
Sep 18 20:01:00.286154 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:00.286128    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f75758d_a29f_4ec6_b39e_73a4ed1a3c73.slice/crio-4b595b9e08265f9ed44c7f84267ba8cc0a3623fa4431cec36c1a2aa03b40b478 WatchSource:0}: Error finding container 4b595b9e08265f9ed44c7f84267ba8cc0a3623fa4431cec36c1a2aa03b40b478: Status 404 returned error can't find the container with id 4b595b9e08265f9ed44c7f84267ba8cc0a3623fa4431cec36c1a2aa03b40b478
Sep 18 20:01:00.287638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.287545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-67q5n"
Sep 18 20:01:00.293515 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.293497    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-79qgh"
Sep 18 20:01:00.297371 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:00.297350    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb69fb34b_9443_4cea_9b7c_19ba8258730a.slice/crio-07edf3d0c6d3614a8240201c552925f2cee4500beb7149d2d35932684c189ad1 WatchSource:0}: Error finding container 07edf3d0c6d3614a8240201c552925f2cee4500beb7149d2d35932684c189ad1: Status 404 returned error can't find the container with id 07edf3d0c6d3614a8240201c552925f2cee4500beb7149d2d35932684c189ad1
Sep 18 20:01:00.300656 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.300635    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:00.311490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.311456    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:00.319307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.319285    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7h7pk"
Sep 18 20:01:00.327932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.327912    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-5bwtn"
Sep 18 20:01:00.334648 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.334624    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-7r7ct"
Sep 18 20:01:00.600942 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.600858    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:00.600942 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.600901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:00.601174 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.601036    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:00.601174 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.601051    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:00.601174 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.601124    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:01.601104655 +0000 UTC m=+5.730406600 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:00.601174 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.601155    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:01.601137153 +0000 UTC m=+5.730439086 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:00.701806 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:00.701771    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:00.701997 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.701974    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:00.702046 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.702005    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:00.702046 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.702019    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:00.702138 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:00.702115    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:01.702095019 +0000 UTC m=+5.831396949 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:01.022499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.022467    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:56:00 +0000 UTC" deadline="2028-03-18 02:06:53.983724094 +0000 UTC"
Sep 18 20:01:01.022499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.022495    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13110h5m52.961233374s"
Sep 18 20:01:01.085180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.085144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-67q5n" event={"ID":"b69fb34b-9443-4cea-9b7c-19ba8258730a","Type":"ContainerStarted","Data":"07edf3d0c6d3614a8240201c552925f2cee4500beb7149d2d35932684c189ad1"}
Sep 18 20:01:01.086415 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.086380    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5dsjh" event={"ID":"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73","Type":"ContainerStarted","Data":"4b595b9e08265f9ed44c7f84267ba8cc0a3623fa4431cec36c1a2aa03b40b478"}
Sep 18 20:01:01.087584 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.087558    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd" event={"ID":"e5266fe9-cd4f-4363-9255-b951f4a56d59","Type":"ContainerStarted","Data":"88ede0c1fb75b381e4a43f967e78a4c53591c48509cd6cf55131683b22a8c28a"}
Sep 18 20:01:01.323098 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.323073    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc367a973_1121_4c33_895f_e2b2f4e60b0f.slice/crio-33cf948ddc821b5927073cf0128143bb41b7d1001c0f60f5737e4f2d757fb5dd WatchSource:0}: Error finding container 33cf948ddc821b5927073cf0128143bb41b7d1001c0f60f5737e4f2d757fb5dd: Status 404 returned error can't find the container with id 33cf948ddc821b5927073cf0128143bb41b7d1001c0f60f5737e4f2d757fb5dd
Sep 18 20:01:01.324047 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.323982    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7edd8626_6905_4132_889b_d2f5043c018e.slice/crio-4265fd267b55ceead996faee9c6185b168b423fff7b4f6890bd7305500377f99 WatchSource:0}: Error finding container 4265fd267b55ceead996faee9c6185b168b423fff7b4f6890bd7305500377f99: Status 404 returned error can't find the container with id 4265fd267b55ceead996faee9c6185b168b423fff7b4f6890bd7305500377f99
Sep 18 20:01:01.326940 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.326916    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode05ba3d9_56f2_4a3b_89b2_4511ace2afbf.slice/crio-3c1c28e7ad90d7a579f07ee986c06a7355a2c40675125f6d1b3f7237c38f9001 WatchSource:0}: Error finding container 3c1c28e7ad90d7a579f07ee986c06a7355a2c40675125f6d1b3f7237c38f9001: Status 404 returned error can't find the container with id 3c1c28e7ad90d7a579f07ee986c06a7355a2c40675125f6d1b3f7237c38f9001
Sep 18 20:01:01.328042 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.328018    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod85665792_b3e0_4b6a_b7f2_cab42e82cd62.slice/crio-cc1c6d9354941785977be417f321a5ad34c5bb70a1fc431f2ac5fce4c1c32969 WatchSource:0}: Error finding container cc1c6d9354941785977be417f321a5ad34c5bb70a1fc431f2ac5fce4c1c32969: Status 404 returned error can't find the container with id cc1c6d9354941785977be417f321a5ad34c5bb70a1fc431f2ac5fce4c1c32969
Sep 18 20:01:01.329146 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.329009    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode10a48b8_18a8_415e_8240_5072512f4f1f.slice/crio-a1f3457ce1ed570cbf7a2b9919e85e3398b8197e58d85ffccacb1471197e7711 WatchSource:0}: Error finding container a1f3457ce1ed570cbf7a2b9919e85e3398b8197e58d85ffccacb1471197e7711: Status 404 returned error can't find the container with id a1f3457ce1ed570cbf7a2b9919e85e3398b8197e58d85ffccacb1471197e7711
Sep 18 20:01:01.330445 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:01.330261    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c0aac5c_761f_4bf0_a9fc_ed9dd9c86755.slice/crio-8f2096b513ec9d310617232adf8a28281e916721b54f8829a6e358161748fd3f WatchSource:0}: Error finding container 8f2096b513ec9d310617232adf8a28281e916721b54f8829a6e358161748fd3f: Status 404 returned error can't find the container with id 8f2096b513ec9d310617232adf8a28281e916721b54f8829a6e358161748fd3f
Sep 18 20:01:01.608580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.608377    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:01.608689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.608609    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:01.609162 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.608537    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:01.609162 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.608756    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:01.609162 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.608829    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:03.608808474 +0000 UTC m=+7.738110410 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:01.609162 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.608891    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:03.608874167 +0000 UTC m=+7.738176099 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:01.709396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:01.709354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:01.709571 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.709509    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:01.709571 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.709535    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:01.709571 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.709550    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:01.709733 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:01.709614    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:03.709594795 +0000 UTC m=+7.838896726 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:02.023269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.023202    2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-17 19:56:00 +0000 UTC" deadline="2028-05-06 13:49:28.80053443 +0000 UTC"
Sep 18 20:01:02.023269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.023268    2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14297h48m26.77727054s"
Sep 18 20:01:02.076855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.076823    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:02.077017 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:02.076954    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:02.077645 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.077369    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:02.077645 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:02.077460    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:02.077645 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.077520    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:02.077645 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:02.077591    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:02.099537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.099506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7h7pk" event={"ID":"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755","Type":"ContainerStarted","Data":"8f2096b513ec9d310617232adf8a28281e916721b54f8829a6e358161748fd3f"}
Sep 18 20:01:02.101732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.101683    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-79qgh" event={"ID":"c367a973-1121-4c33-895f-e2b2f4e60b0f","Type":"ContainerStarted","Data":"33cf948ddc821b5927073cf0128143bb41b7d1001c0f60f5737e4f2d757fb5dd"}
Sep 18 20:01:02.106827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.106769    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal" event={"ID":"857cbacc60cccfe8f3eccb86ceb6ba23","Type":"ContainerStarted","Data":"35d48ba4abc0a14b326ecbe95e463cce64c34c3db7cbd7e8e70c9f40d8c2e772"}
Sep 18 20:01:02.118733 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.118502    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"a1f3457ce1ed570cbf7a2b9919e85e3398b8197e58d85ffccacb1471197e7711"}
Sep 18 20:01:02.120325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.120119    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-131-20.ec2.internal" podStartSLOduration=3.1201038 podStartE2EDuration="3.1201038s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:02.119885006 +0000 UTC m=+6.249186960" watchObservedRunningTime="2026-09-18 20:01:02.1201038 +0000 UTC m=+6.249405751"
Sep 18 20:01:02.124490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.124462    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mvnvn" event={"ID":"85665792-b3e0-4b6a-b7f2-cab42e82cd62","Type":"ContainerStarted","Data":"cc1c6d9354941785977be417f321a5ad34c5bb70a1fc431f2ac5fce4c1c32969"}
Sep 18 20:01:02.129973 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.129949    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerStarted","Data":"3c1c28e7ad90d7a579f07ee986c06a7355a2c40675125f6d1b3f7237c38f9001"}
Sep 18 20:01:02.132228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.132104    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bwtn" event={"ID":"7edd8626-6905-4132-889b-d2f5043c018e","Type":"ContainerStarted","Data":"4265fd267b55ceead996faee9c6185b168b423fff7b4f6890bd7305500377f99"}
Sep 18 20:01:02.148221 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.148194    2566 generic.go:356] "Generic (PLEG): container finished" podID="bc3eefcfaadb741284b07feb390941dd" containerID="b9cca1bd3694cc4cdac472bea310684e5dd85cd062f087c51272ec1c37537e2e" exitCode=0
Sep 18 20:01:02.148321 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:02.148235    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal" event={"ID":"bc3eefcfaadb741284b07feb390941dd","Type":"ContainerDied","Data":"b9cca1bd3694cc4cdac472bea310684e5dd85cd062f087c51272ec1c37537e2e"}
Sep 18 20:01:03.174169 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:03.174122    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal" event={"ID":"bc3eefcfaadb741284b07feb390941dd","Type":"ContainerStarted","Data":"e93d000ea820959d9cf2973f19c2024d38865dbc5d6cc1f69b8cf71d72c67d69"}
Sep 18 20:01:03.189354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:03.189073    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-131-20.ec2.internal" podStartSLOduration=4.189038229 podStartE2EDuration="4.189038229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:03.18829814 +0000 UTC m=+7.317600083" watchObservedRunningTime="2026-09-18 20:01:03.189038229 +0000 UTC m=+7.318340182"
Sep 18 20:01:03.623722 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:03.623684    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:03.623890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:03.623734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:03.623890 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.623883    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:03.624014 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.623946    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:07.623922908 +0000 UTC m=+11.753224845 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:03.624550 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.624343    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:03.624550 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.624392    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:07.624378216 +0000 UTC m=+11.753680150 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:03.724350 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:03.724314    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:03.724534 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.724515    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:03.724635 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.724541    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:03.724635 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.724553    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:03.724635 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:03.724606    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:07.72458942 +0000 UTC m=+11.853891355 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:04.074506 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:04.074475    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:04.074692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:04.074486    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:04.074692 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:04.074614    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:04.074692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:04.074668    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:04.074855 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:04.074758    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:04.074855 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:04.074840    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:06.075051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:06.075015    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:06.075494 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:06.075186    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:06.075494 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:06.075212    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:06.075494 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:06.075344    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:06.075494 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:06.075381    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:06.075494 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:06.075450    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:07.654396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:07.654355    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:07.654834 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:07.654405    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:07.654834 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.654528    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:07.654834 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.654544    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:07.654834 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.654594    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:15.654574652 +0000 UTC m=+19.783876597 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:07.654834 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.654616    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:15.654606876 +0000 UTC m=+19.783908806 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:07.755652 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:07.755555    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:07.755813 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.755696    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:07.755813 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.755719    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:07.755813 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.755730    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:07.755813 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:07.755788    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:15.755772119 +0000 UTC m=+19.885074054 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:08.075785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:08.075742    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:08.075942 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:08.075869    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:08.076087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:08.076033    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:08.076205 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:08.076130    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:08.076205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:08.076156    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:08.076331 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:08.076215    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:10.074280 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:10.074154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:10.074710 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:10.074278    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:10.074710 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:10.074154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:10.074710 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:10.074154    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:10.074710 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:10.074399    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:10.074710 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:10.074493    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:12.074530 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:12.074489    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:12.074974 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:12.074490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:12.074974 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:12.074624    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:12.074974 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:12.074501    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:12.074974 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:12.074696    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:12.074974 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:12.074764    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:14.074828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:14.074791    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:14.075297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:14.074840    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:14.075297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:14.074789    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:14.075297 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:14.074924    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:14.075297 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:14.075011    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:14.075297 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:14.075111    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:15.712159 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:15.712126    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:15.712159 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:15.712165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:15.712616 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.712284    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:15.712616 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.712342    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:31.712328993 +0000 UTC m=+35.841630922 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:15.712616 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.712284    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:15.712616 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.712413    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:31.712400104 +0000 UTC m=+35.841702037 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:15.813088 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:15.813039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:15.813252 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.813214    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:15.813252 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.813241    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:15.813252 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.813255    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:15.813389 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:15.813311    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:31.813292149 +0000 UTC m=+35.942594078 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:16.074180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:16.074106    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:16.074341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:16.074106    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:16.074341 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:16.074241    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:16.074341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:16.074107    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:16.074341 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:16.074304    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:16.074533 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:16.074386    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:18.075199 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:18.075171    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:18.075604 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:18.075267    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:18.075604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:18.075279    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:18.075604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:18.075300    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:18.075604 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:18.075360    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:18.075604 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:18.075450    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:19.209040 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.208616    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-79qgh" event={"ID":"c367a973-1121-4c33-895f-e2b2f4e60b0f","Type":"ContainerStarted","Data":"69400d3513946e3c0d8763db9b21f09a0437ff55348b29c3bd4859a88f736df1"}
Sep 18 20:01:19.210369 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.210339    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5dsjh" event={"ID":"1f75758d-a29f-4ec6-b39e-73a4ed1a3c73","Type":"ContainerStarted","Data":"f57b43b0abf7e3841e35c628a344a1856e8ed22da3608dfb18a4e1d02730f687"}
Sep 18 20:01:19.213701 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.213680    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:01:19.214133 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214098    2566 generic.go:356] "Generic (PLEG): container finished" podID="e10a48b8-18a8-415e-8240-5072512f4f1f" containerID="bd47b1ebddb21ddc8039ce016e8a74cb7d3025d134e07c68f54d4043721c54e4" exitCode=1
Sep 18 20:01:19.214234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"349c408b21e0b709bd42ed8a47e0753648525127a30a013612ed3bf129763ea7"}
Sep 18 20:01:19.214234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"2fa834d35bfad27ec57daad44e0a4a4f600acf6fb55a8f7b82d48316ef2ffa41"}
Sep 18 20:01:19.214337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214243    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"68509911a3da6ee9319ffe594a139d860601db12e6a2add1dbe35cfc01059c4b"}
Sep 18 20:01:19.214337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214287    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"89f92e35a078276ff2a2f262a022f3d26670762af9b0bf4935b56293aa90256b"}
Sep 18 20:01:19.214337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerDied","Data":"bd47b1ebddb21ddc8039ce016e8a74cb7d3025d134e07c68f54d4043721c54e4"}
Sep 18 20:01:19.214432 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.214351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"d7ec5db84fde06ce9146a064498b46d6ac0018ef52414360293473c534bb4607"}
Sep 18 20:01:19.215817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.215791    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-mvnvn" event={"ID":"85665792-b3e0-4b6a-b7f2-cab42e82cd62","Type":"ContainerStarted","Data":"6e3c15463d175886ffd3ab07289ba059fd78ae17181653baa1f2a10ec146fed8"}
Sep 18 20:01:19.217514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.217492    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="8640018b99eabb8d493c701dbdeb7f4f1ae3d9c29be4130465f40b8cddb7c8b5" exitCode=0
Sep 18 20:01:19.217603 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.217583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"8640018b99eabb8d493c701dbdeb7f4f1ae3d9c29be4130465f40b8cddb7c8b5"}
Sep 18 20:01:19.219681 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.219656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-67q5n" event={"ID":"b69fb34b-9443-4cea-9b7c-19ba8258730a","Type":"ContainerStarted","Data":"a785407c6812e26893fc32b376bb3803f51c8d98a819bb648c022e51e7ae0854"}
Sep 18 20:01:19.221386 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.221359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd" event={"ID":"e5266fe9-cd4f-4363-9255-b951f4a56d59","Type":"ContainerStarted","Data":"4f8b6741b7d3e033e6542191f823cc51f0b154c4883bc57fe986e56d8ecf99ac"}
Sep 18 20:01:19.223535 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.223486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7h7pk" event={"ID":"5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755","Type":"ContainerStarted","Data":"f2fb3b0932538086606a9cf922bcdf2fd5a2795e75f783f533ca00070fb9760e"}
Sep 18 20:01:19.227002 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.226966    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-79qgh" podStartSLOduration=4.302023866 podStartE2EDuration="21.226954835s" totalImagesPullingTime="16.924930969s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:19.22671109 +0000 UTC m=+23.356013042" watchObservedRunningTime="2026-09-18 20:01:19.226954835 +0000 UTC m=+23.356256786"
Sep 18 20:01:19.259049 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.259002    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-67q5n" podStartSLOduration=3.376315572 podStartE2EDuration="21.258988899s" totalImagesPullingTime="17.882673327s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:19.258635952 +0000 UTC m=+23.387937901" watchObservedRunningTime="2026-09-18 20:01:19.258988899 +0000 UTC m=+23.388290850"
Sep 18 20:01:19.296089 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.296009    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-mvnvn" podStartSLOduration=4.444103928 podStartE2EDuration="21.295987113s" totalImagesPullingTime="16.851883185s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:19.295520584 +0000 UTC m=+23.424822534" watchObservedRunningTime="2026-09-18 20:01:19.295987113 +0000 UTC m=+23.425289065"
Sep 18 20:01:19.296241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.296197    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5dsjh" podStartSLOduration=3.402720388 podStartE2EDuration="21.296190779s" totalImagesPullingTime="17.893470391s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:19.277518979 +0000 UTC m=+23.406820929" watchObservedRunningTime="2026-09-18 20:01:19.296190779 +0000 UTC m=+23.425492729"
Sep 18 20:01:19.317764 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.317722    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7h7pk" podStartSLOduration=4.468081637 podStartE2EDuration="21.317707227s" totalImagesPullingTime="16.84962559s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:19.317097809 +0000 UTC m=+23.446399760" watchObservedRunningTime="2026-09-18 20:01:19.317707227 +0000 UTC m=+23.447009211"
Sep 18 20:01:19.880512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:19.880490    2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 18 20:01:20.065008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.064861    2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-18T20:01:19.880508858Z","UUID":"91904aec-82d0-4372-a784-fcb3c847acab","Handler":null,"Name":"","Endpoint":""}
Sep 18 20:01:20.066465 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.066422    2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 18 20:01:20.066465 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.066468    2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 18 20:01:20.074355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.074329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:20.074474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.074329    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:20.074474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.074461    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:20.074592 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:20.074457    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:20.074646 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:20.074594    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:20.074713 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:20.074686    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:20.228437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.228407    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd" event={"ID":"e5266fe9-cd4f-4363-9255-b951f4a56d59","Type":"ContainerStarted","Data":"cb7d441cba89b4e94205e399869ce24f2d841ac461d116ca1bc8286c69f6250b"}
Sep 18 20:01:20.230162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.230134    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-5bwtn" event={"ID":"7edd8626-6905-4132-889b-d2f5043c018e","Type":"ContainerStarted","Data":"e96d88d29b3297d580aa08692cc5e344c7a845c2f024395fbc591d2b17b59d6a"}
Sep 18 20:01:20.244127 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:20.244090    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-5bwtn" podStartSLOduration=5.388506486 podStartE2EDuration="22.244077215s" totalImagesPullingTime="16.855570729s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:20.243854266 +0000 UTC m=+24.373156217" watchObservedRunningTime="2026-09-18 20:01:20.244077215 +0000 UTC m=+24.373379165"
Sep 18 20:01:21.235627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:21.235599    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:01:21.236115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:21.236087    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"b37975d758cfad66ccf3f2d41d9e6e456e87bfd2f21094aca5bbaafcbd3847f7"}
Sep 18 20:01:22.074018 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:22.073983    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:22.074216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:22.073988    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:22.074216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:22.073988    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:22.074216 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:22.074152    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:22.074369 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:22.074329    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:22.074448 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:22.074423    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:22.241387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:22.241351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd" event={"ID":"e5266fe9-cd4f-4363-9255-b951f4a56d59","Type":"ContainerStarted","Data":"4e440d9610a608d7667c9ba823c95287fe2d21d7190ac9d7ec03dbe65a8e9731"}
Sep 18 20:01:22.258677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:22.258635    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-mlrbd" podStartSLOduration=3.575864301 podStartE2EDuration="24.258621091s" totalImagesPullingTime="20.68275679s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:22.257441439 +0000 UTC m=+26.386743390" watchObservedRunningTime="2026-09-18 20:01:22.258621091 +0000 UTC m=+26.387923041"
Sep 18 20:01:23.476462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:23.476275    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:23.476909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:23.476853    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:24.074541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.074505    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:24.074726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.074509    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:24.074726 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:24.074631    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:24.074726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.074642    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:24.074869 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:24.074759    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:24.074908 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:24.074874    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:24.250007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.249985    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:01:24.250472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.250444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"437451a674b34e85a2343cd8b75ce4988c4b2e33b928bc42bedbfbafe5dff619"}
Sep 18 20:01:24.250957 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:24.250940    2566 scope.go:117] "RemoveContainer" containerID="bd47b1ebddb21ddc8039ce016e8a74cb7d3025d134e07c68f54d4043721c54e4"
Sep 18 20:01:25.219352 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.219326    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:25.220023 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.219785    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-mvnvn"
Sep 18 20:01:25.255589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.255567    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:01:25.255924 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.255904    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" event={"ID":"e10a48b8-18a8-415e-8240-5072512f4f1f","Type":"ContainerStarted","Data":"969366530feb9228226a038a35a1aeda91727ea7e752c604862558b797fa99fd"}
Sep 18 20:01:25.256009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.255992    2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness"
Sep 18 20:01:25.256371 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.256336    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:25.256465 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.256423    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:25.257891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.257867    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="1d2991be38cdfffc6f17e00ce4699f15e5531243448905f75e283851632dcc69" exitCode=0
Sep 18 20:01:25.257988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.257950    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"1d2991be38cdfffc6f17e00ce4699f15e5531243448905f75e283851632dcc69"}
Sep 18 20:01:25.271467 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.271442    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:25.271553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.271537    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:25.284179 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:25.284142    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx" podStartSLOduration=10.394357581 podStartE2EDuration="27.284126629s" totalImagesPullingTime="16.889769048s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:25.282830558 +0000 UTC m=+29.412132509" watchObservedRunningTime="2026-09-18 20:01:25.284126629 +0000 UTC m=+29.413428579"
Sep 18 20:01:26.012590 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.012417    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:01:26.047342 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.047317    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zbm8h"]
Sep 18 20:01:26.047452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.047437    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:26.047575 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:26.047548    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:26.051273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.051248    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jr7x"]
Sep 18 20:01:26.051355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.051299    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k546r"]
Sep 18 20:01:26.051409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.051362    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:26.051475 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:26.051452    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:26.051527 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.051478    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:26.051573 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:26.051550    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:26.261604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.261571    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="d31f747f6ccba0d14cc7020dfb51e5bc1b671665c7e35708deaf57bed01600cd" exitCode=0
Sep 18 20:01:26.261931 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:26.261653    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"d31f747f6ccba0d14cc7020dfb51e5bc1b671665c7e35708deaf57bed01600cd"}
Sep 18 20:01:27.074703 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:27.074676    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:27.074809 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:27.074782    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:27.266589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:27.266556    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="54a15e79f5f811f5a1096a38fe3296fabb879f467c7e17956bb1cefa422afb6e" exitCode=0
Sep 18 20:01:27.267084 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:27.266631    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"54a15e79f5f811f5a1096a38fe3296fabb879f467c7e17956bb1cefa422afb6e"}
Sep 18 20:01:28.075257 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:28.075231    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:28.075449 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:28.075287    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:28.075449 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:28.075371    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:28.075576 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:28.075436    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:29.074341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:29.074307    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:29.074844 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:29.074437    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:30.078329 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:30.078301    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:30.078736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:30.078302    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:30.078736 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:30.078405    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-k546r" podUID="fee2e6d4-8e67-4337-bae7-abebdf14f750"
Sep 18 20:01:30.078736 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:30.078492    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-zbm8h" podUID="1abab961-0419-4eb6-b08f-78d713072033"
Sep 18 20:01:31.074429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.074393    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:31.074652 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.074524    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:01:31.285300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.285272    2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-131-20.ec2.internal" event="NodeReady"
Sep 18 20:01:31.285678 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.285418    2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 18 20:01:31.321770 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.321737    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:01:31.325656 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.325623    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"]
Sep 18 20:01:31.325785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.325717    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.328519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.328492    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
Sep 18 20:01:31.328626 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.328531    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
Sep 18 20:01:31.328626 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.328544    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.328626 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.328556    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t5jb8\""
Sep 18 20:01:31.328626 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.328556    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
Sep 18 20:01:31.331261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.331193    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
Sep 18 20:01:31.331261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.331216    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
Sep 18 20:01:31.331609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.331581    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vl2fh\""
Sep 18 20:01:31.335257 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.334641    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
Sep 18 20:01:31.343292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.343271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"]
Sep 18 20:01:31.344166 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.344147    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:01:31.346976 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.346951    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qvdjd"]
Sep 18 20:01:31.352677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.352653    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-zrnwv"]
Sep 18 20:01:31.352785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.352769    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.356008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.355985    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 18 20:01:31.356109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.355986    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw2ht\""
Sep 18 20:01:31.356164 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.355989    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 18 20:01:31.356438 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.356419    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.358404 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.358381    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 18 20:01:31.358574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.358547    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-xpvnc\""
Sep 18 20:01:31.358574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.358563    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 18 20:01:31.358802 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.358787    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 18 20:01:31.361028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.361006    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrnwv"]
Sep 18 20:01:31.361790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.361768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qvdjd"]
Sep 18 20:01:31.425752 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c397f480-13a7-4cd9-8dd5-690d317bfb4d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.425897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425763    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.425897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425791    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5be920-996d-492f-8de2-83ab8c7b1d30-config-volume\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.425897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425812    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.425897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425861    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426091 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425934    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426091 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.425971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426091 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426033    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.426222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426105    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426134    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426156    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6k7jw\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.426222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426195    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5be920-996d-492f-8de2-83ab8c7b1d30-tmp-dir\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.426222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426212    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vjcrv\" (UniqueName: \"kubernetes.io/projected/bd5be920-996d-492f-8de2-83ab8c7b1d30-kube-api-access-vjcrv\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.426442 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.426298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.492589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.492506    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mvnvn_85665792-b3e0-4b6a-b7f2-cab42e82cd62/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:31.527532 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527492    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527542    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527594    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527628    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527650    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527671    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6k7jw\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527713    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5be920-996d-492f-8de2-83ab8c7b1d30-tmp-dir\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527733    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vjcrv\" (UniqueName: \"kubernetes.io/projected/bd5be920-996d-492f-8de2-83ab8c7b1d30-kube-api-access-vjcrv\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.527750    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527765    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.527825    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:01:32.027801828 +0000 UTC m=+36.157103761 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527850    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d72cl\" (UniqueName: \"kubernetes.io/projected/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-kube-api-access-d72cl\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.527861    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.527874    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527895    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c397f480-13a7-4cd9-8dd5-690d317bfb4d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527942    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.527998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.527989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5be920-996d-492f-8de2-83ab8c7b1d30-config-volume\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.528526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528021    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.528526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528050    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-cert\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.528526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528215    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.528526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528492    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bd5be920-996d-492f-8de2-83ab8c7b1d30-tmp-dir\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.528526 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.528507    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:31.528767 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.528554    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:32.028537445 +0000 UTC m=+36.157839379 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:31.528767 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.528580    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:32.028569964 +0000 UTC m=+36.157871907 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:31.528767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528603    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.528767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.528980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528864    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c397f480-13a7-4cd9-8dd5-690d317bfb4d-nginx-conf\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:31.528980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.528876    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bd5be920-996d-492f-8de2-83ab8c7b1d30-config-volume\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.532073 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.532035    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.532193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.532050    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.539804 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.539778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.543966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.543944    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vjcrv\" (UniqueName: \"kubernetes.io/projected/bd5be920-996d-492f-8de2-83ab8c7b1d30-kube-api-access-vjcrv\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:31.548097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.548029    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6k7jw\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:31.587407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.587349    2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-mvnvn_85665792-b3e0-4b6a-b7f2-cab42e82cd62/konnectivity-agent/0.log" limit=1024
Sep 18 20:01:31.628887 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.628857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d72cl\" (UniqueName: \"kubernetes.io/projected/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-kube-api-access-d72cl\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.629047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.628901    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-cert\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.631540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.631519    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-cert\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.636317 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.636297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d72cl\" (UniqueName: \"kubernetes.io/projected/a6e4fdd6-d434-4c3f-af0b-0f7531647a85-kube-api-access-d72cl\") pod \"ingress-canary-zrnwv\" (UID: \"a6e4fdd6-d434-4c3f-af0b-0f7531647a85\") " pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.671611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.671581    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-zrnwv"
Sep 18 20:01:31.729609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.729581    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:31.729792 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.729620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:31.729792 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.729746    2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:31.729792 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.729747    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:31.729954 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.729814    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.729796953 +0000 UTC m=+67.859098883 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 18 20:01:31.729954 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.729833    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret podName:1abab961-0419-4eb6-b08f-78d713072033 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.729823056 +0000 UTC m=+67.859124989 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret") pod "global-pull-secret-syncer-zbm8h" (UID: "1abab961-0419-4eb6-b08f-78d713072033") : object "kube-system"/"original-pull-secret" not registered
Sep 18 20:01:31.830914 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:31.830831    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:31.831101 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.831032    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 18 20:01:31.831101 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.831078    2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 18 20:01:31.831101 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.831091    2566 projected.go:196] Error preparing data for projected volume kube-api-access-jfwtd for pod openshift-network-diagnostics/network-check-target-k546r: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:31.831265 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:31.831158    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd podName:fee2e6d4-8e67-4337-bae7-abebdf14f750 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.831139446 +0000 UTC m=+67.960441398 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jfwtd" (UniqueName: "kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd") pod "network-check-target-k546r" (UID: "fee2e6d4-8e67-4337-bae7-abebdf14f750") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 18 20:01:32.032896 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.032696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:32.033094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.032948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:32.033094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.032998    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:32.033094 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.032873    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:32.033094 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033041    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:32.033273 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033015    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:32.033273 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033112    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:33.033092465 +0000 UTC m=+37.162394408 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:32.033273 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033122    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:32.033273 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033149    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:33.033135044 +0000 UTC m=+37.162436980 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:32.033273 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:32.033177    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:01:33.033158662 +0000 UTC m=+37.162460605 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:32.074006 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.073975    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:01:32.074185 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.074149    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:01:32.076430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.076404    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:01:32.076548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.076431    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:01:32.076717 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.076672    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:01:32.076862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.076842    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4pmvp\""
Sep 18 20:01:32.602007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:32.601984    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-zrnwv"]
Sep 18 20:01:32.653917 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:01:32.653894    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6e4fdd6_d434_4c3f_af0b_0f7531647a85.slice/crio-fcb73102d838bb04a4de18b743d3c5cb6fbd6b2b4987e145afde425b641e09ef WatchSource:0}: Error finding container fcb73102d838bb04a4de18b743d3c5cb6fbd6b2b4987e145afde425b641e09ef: Status 404 returned error can't find the container with id fcb73102d838bb04a4de18b743d3c5cb6fbd6b2b4987e145afde425b641e09ef
Sep 18 20:01:33.040365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.040333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:33.040521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.040378    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:33.040521 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040470    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:33.040521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.040485    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:33.040521 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040520    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:33.040671 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040582    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:01:35.040560879 +0000 UTC m=+39.169862822 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:33.040671 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040602    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:35.040594119 +0000 UTC m=+39.169896052 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:33.040671 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040611    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:33.040671 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040623    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:33.040671 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:33.040656    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:35.040648961 +0000 UTC m=+39.169950889 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:33.074691 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.074660    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:01:33.077105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.077087    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms6pd\""
Sep 18 20:01:33.077223 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.077086    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:01:33.285106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.285073    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="f1d8c2aa69d52fb5faf59693fee46f4c4f555d457a876a2964cc2d20cf52875c" exitCode=0
Sep 18 20:01:33.285283 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.285170    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"f1d8c2aa69d52fb5faf59693fee46f4c4f555d457a876a2964cc2d20cf52875c"}
Sep 18 20:01:33.286846 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:33.286813    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrnwv" event={"ID":"a6e4fdd6-d434-4c3f-af0b-0f7531647a85","Type":"ContainerStarted","Data":"fcb73102d838bb04a4de18b743d3c5cb6fbd6b2b4987e145afde425b641e09ef"}
Sep 18 20:01:34.293502 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:34.293462    2566 generic.go:356] "Generic (PLEG): container finished" podID="e05ba3d9-56f2-4a3b-89b2-4511ace2afbf" containerID="eb8fd358f5b7e464189e0ef6cc6fdf169001c541db9ef016d1c996ff5f30740f" exitCode=0
Sep 18 20:01:34.294105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:34.293515    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerDied","Data":"eb8fd358f5b7e464189e0ef6cc6fdf169001c541db9ef016d1c996ff5f30740f"}
Sep 18 20:01:35.057307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:35.057271    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:35.057459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:35.057343    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:35.057459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:35.057390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:35.057459 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057428    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:35.057459 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057445    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:35.057602 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057496    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:35.057602 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057499    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:35.057602 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057503    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:39.057485889 +0000 UTC m=+43.186787835 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:35.057907 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057606    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:39.057581239 +0000 UTC m=+43.186883177 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:35.057907 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:35.057637    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:01:39.057624066 +0000 UTC m=+43.186925995 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:35.298813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:35.298541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" event={"ID":"e05ba3d9-56f2-4a3b-89b2-4511ace2afbf","Type":"ContainerStarted","Data":"1f105235f8d88cef50f0c7c3d4fd8f8673b16c8c19fbcfd2e8f61fd79abbc7f2"}
Sep 18 20:01:35.319284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:35.319199    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-7r7ct" podStartSLOduration=8.776241744 podStartE2EDuration="37.319183036s" totalImagesPullingTime="28.542941292s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:35.318141154 +0000 UTC m=+39.447443104" watchObservedRunningTime="2026-09-18 20:01:35.319183036 +0000 UTC m=+39.448484983"
Sep 18 20:01:36.302110 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:36.302073    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-zrnwv" event={"ID":"a6e4fdd6-d434-4c3f-af0b-0f7531647a85","Type":"ContainerStarted","Data":"73cc345602773ea989c830f70c150ae224a27fa4721630534e7b0275498fa1f2"}
Sep 18 20:01:36.317699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:36.317657    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-zrnwv" podStartSLOduration=2.667367355 podStartE2EDuration="5.317642623s" totalImagesPullingTime="2.650275268s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:01:36.317254218 +0000 UTC m=+40.446556169" watchObservedRunningTime="2026-09-18 20:01:36.317642623 +0000 UTC m=+40.446944580"
Sep 18 20:01:39.090030 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:39.089993    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:39.090052    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:39.090106    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090162    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090181    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090192    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090217    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090234    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:47.090220382 +0000 UTC m=+51.219522311 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090248    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:01:47.090242147 +0000 UTC m=+51.219544075 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:39.090397 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:39.090260    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:01:47.090254114 +0000 UTC m=+51.219556043 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:47.150500 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:47.150460    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:47.150518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:47.150579    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150618    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150674    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150682    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150685    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.150666336 +0000 UTC m=+67.279968273 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150685    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150755    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.150737071 +0000 UTC m=+67.280039007 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:01:47.150971 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:01:47.150794    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:03.150781342 +0000 UTC m=+67.280083285 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:01:57.276843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:01:57.276814    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-5xrvx"
Sep 18 20:02:03.168383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.168345    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:02:03.168383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.168390    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168492    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168495    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.168552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168573    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:35.168550176 +0000 UTC m=+99.297852136 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168597    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:02:35.16858796 +0000 UTC m=+99.297889891 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168615    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168631    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:02:03.168859 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.168672    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:02:35.168660329 +0000 UTC m=+99.297962267 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:02:03.773983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.773948    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:02:03.774168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.773999    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:02:03.776712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.776692    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 18 20:02:03.776771 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.776697    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 18 20:02:03.784705 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.784689    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 20:02:03.784774 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:03.784752    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:03:07.784735234 +0000 UTC m=+131.914037168 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : secret "metrics-daemon-secret" not found
Sep 18 20:02:03.787663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.787643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/1abab961-0419-4eb6-b08f-78d713072033-original-pull-secret\") pod \"global-pull-secret-syncer-zbm8h\" (UID: \"1abab961-0419-4eb6-b08f-78d713072033\") " pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:02:03.874846 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.874813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:02:03.877419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.877402    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 18 20:02:03.885123 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.885099    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-zbm8h"
Sep 18 20:02:03.886900 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.886886    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 18 20:02:03.897833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:03.897815    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jfwtd\" (UniqueName: \"kubernetes.io/projected/fee2e6d4-8e67-4337-bae7-abebdf14f750-kube-api-access-jfwtd\") pod \"network-check-target-k546r\" (UID: \"fee2e6d4-8e67-4337-bae7-abebdf14f750\") " pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:02:04.183801 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:04.183710    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-zbm8h"]
Sep 18 20:02:04.188251 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:02:04.188219    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1abab961_0419_4eb6_b08f_78d713072033.slice/crio-5ea3326547635e6c68a9a94393df72679af436094f57042ba202635d6ffd0394 WatchSource:0}: Error finding container 5ea3326547635e6c68a9a94393df72679af436094f57042ba202635d6ffd0394: Status 404 returned error can't find the container with id 5ea3326547635e6c68a9a94393df72679af436094f57042ba202635d6ffd0394
Sep 18 20:02:04.193541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:04.193523    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4pmvp\""
Sep 18 20:02:04.202269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:04.202250    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:02:04.366230 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:04.366190    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zbm8h" event={"ID":"1abab961-0419-4eb6-b08f-78d713072033","Type":"ContainerStarted","Data":"5ea3326547635e6c68a9a94393df72679af436094f57042ba202635d6ffd0394"}
Sep 18 20:02:04.495435 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:04.495402    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-k546r"]
Sep 18 20:02:04.498845 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:02:04.498822    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfee2e6d4_8e67_4337_bae7_abebdf14f750.slice/crio-d13543ee9662227956cf96802e1ba993cfd3b1151b662b1e7ea8b44e56b035f7 WatchSource:0}: Error finding container d13543ee9662227956cf96802e1ba993cfd3b1151b662b1e7ea8b44e56b035f7: Status 404 returned error can't find the container with id d13543ee9662227956cf96802e1ba993cfd3b1151b662b1e7ea8b44e56b035f7
Sep 18 20:02:05.370975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:05.370909    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k546r" event={"ID":"fee2e6d4-8e67-4337-bae7-abebdf14f750","Type":"ContainerStarted","Data":"d13543ee9662227956cf96802e1ba993cfd3b1151b662b1e7ea8b44e56b035f7"}
Sep 18 20:02:09.383379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:09.383291    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-zbm8h" event={"ID":"1abab961-0419-4eb6-b08f-78d713072033","Type":"ContainerStarted","Data":"a2118f75af33d262c0a0780bf03af3971f4fe80bc56823d2b575b66f0b9ccdaa"}
Sep 18 20:02:09.384775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:09.384743    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-k546r" event={"ID":"fee2e6d4-8e67-4337-bae7-abebdf14f750","Type":"ContainerStarted","Data":"7307a95fb476473662c5a1aea91fe306beeb48b251267393dd923359ace4b570"}
Sep 18 20:02:09.384882 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:09.384855    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:02:09.399208 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:09.399156    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-zbm8h" podStartSLOduration=66.56998552 podStartE2EDuration="1m11.399140526s" totalImagesPullingTime="4.829155006s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:02:09.398866848 +0000 UTC m=+73.528168798" watchObservedRunningTime="2026-09-18 20:02:09.399140526 +0000 UTC m=+73.528442477"
Sep 18 20:02:09.413313 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:09.413274    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-k546r" podStartSLOduration=66.898673252 podStartE2EDuration="1m11.413258654s" totalImagesPullingTime="4.514585402s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:02:09.412729568 +0000 UTC m=+73.542031517" watchObservedRunningTime="2026-09-18 20:02:09.413258654 +0000 UTC m=+73.542560605"
Sep 18 20:02:35.202716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:35.202658    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:02:35.202716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:35.202730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.202804    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.202809    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.202856    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:03:39.202841062 +0000 UTC m=+163.332142991 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.202941    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:03:39.202911183 +0000 UTC m=+163.332213120 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:35.203000    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.203095    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.203104    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:02:35.203371 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:02:35.203160    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:03:39.203151197 +0000 UTC m=+163.332453126 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:02:40.389790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:02:40.389757    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-k546r"
Sep 18 20:03:07.817134 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:07.817092    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:03:07.817592 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:07.817255    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 20:03:07.817592 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:07.817327    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:09.817311627 +0000 UTC m=+253.946613556 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : secret "metrics-daemon-secret" not found
Sep 18 20:03:34.338384 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:34.338337    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1"
Sep 18 20:03:34.345455 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:34.345436    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8" podUID="c397f480-13a7-4cd9-8dd5-690d317bfb4d"
Sep 18 20:03:34.364588 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:34.364572    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qvdjd" podUID="bd5be920-996d-492f-8de2-83ab8c7b1d30"
Sep 18 20:03:34.583828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:34.583804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:03:34.583828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:34.583813    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:03:34.583986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:34.583804    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:03:36.082553 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:36.082518    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:03:39.224607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:39.224500    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:03:39.224607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:39.224567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:03:39.224607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:39.224603    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224659    2566 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224707    2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224740    2566 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224759    2566 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-65c945669c-gwd5f: secret "image-registry-tls" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224747    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert podName:c397f480-13a7-4cd9-8dd5-690d317bfb4d nodeName:}" failed. No retries permitted until 2026-09-18 20:05:41.224726443 +0000 UTC m=+285.354028399 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert") pod "networking-console-plugin-6446c8d75d-9csd8" (UID: "c397f480-13a7-4cd9-8dd5-690d317bfb4d") : secret "networking-console-plugin-cert" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224822    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls podName:bd5be920-996d-492f-8de2-83ab8c7b1d30 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:41.224808107 +0000 UTC m=+285.354110041 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls") pod "dns-default-qvdjd" (UID: "bd5be920-996d-492f-8de2-83ab8c7b1d30") : secret "dns-default-metrics-tls" not found
Sep 18 20:03:39.225102 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:03:39.224837    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls podName:a5ab2815-9837-437a-bae0-c11f02acaac1 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:41.22482847 +0000 UTC m=+285.354130399 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls") pod "image-registry-65c945669c-gwd5f" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1") : secret "image-registry-tls" not found
Sep 18 20:03:51.074221 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:03:51.074180    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:05:02.806143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.806107    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"]
Sep 18 20:05:02.808354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.808338    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"
Sep 18 20:05:02.810473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.810457    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-fqdch\""
Sep 18 20:05:02.815446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.815426    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"]
Sep 18 20:05:02.817165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.817147    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-2pxgc"]
Sep 18 20:05:02.817281 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.817266    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:02.818918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.818900    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-94896f94d-jhtw4"]
Sep 18 20:05:02.819032 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.819016    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.819535 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.819515    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
Sep 18 20:05:02.819687 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.819671    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-9pzwc\""
Sep 18 20:05:02.820252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.820218    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
Sep 18 20:05:02.820252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.820242    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.820409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.820343    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.824301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.821825    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
Sep 18 20:05:02.824301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.822878    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
Sep 18 20:05:02.824301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.823185    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.825274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.824795    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"]
Sep 18 20:05:02.825274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.824924    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.825805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.825659    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-s9rwv\""
Sep 18 20:05:02.825805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.825762    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.828141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.828121    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
Sep 18 20:05:02.828858 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.828210    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-nm52h\""
Sep 18 20:05:02.829160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.829143    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
Sep 18 20:05:02.829255 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.829181    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.829482 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.829466    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
Sep 18 20:05:02.829599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.829579    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
Sep 18 20:05:02.829883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.829865    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.830513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.830493    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
Sep 18 20:05:02.832373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.832351    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"]
Sep 18 20:05:02.834625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.834601    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-2pxgc"]
Sep 18 20:05:02.850000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.849974    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-94896f94d-jhtw4"]
Sep 18 20:05:02.903105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903051    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rnxd\" (UniqueName: \"kubernetes.io/projected/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-kube-api-access-8rnxd\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903116    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-stats-auth\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903141    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903170    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-serving-cert\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e39a38af-479b-4c64-91d2-70f12b9525b4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903228    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.903302 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903291    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-tmp\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903313    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqt92\" (UniqueName: \"kubernetes.io/projected/8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d-kube-api-access-xqt92\") pod \"network-check-source-7cbc45fddc-xq5cn\" (UID: \"8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903389    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903457    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtjxh\" (UniqueName: \"kubernetes.io/projected/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-kube-api-access-dtjxh\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903476    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-default-certificate\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903491    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-snapshots\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:02.903542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e39a38af-479b-4c64-91d2-70f12b9525b4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:02.903753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.903551    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-smcbs\" (UniqueName: \"kubernetes.io/projected/e39a38af-479b-4c64-91d2-70f12b9525b4-kube-api-access-smcbs\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:02.905744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.905721    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"]
Sep 18 20:05:02.908230 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.908215    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"
Sep 18 20:05:02.911119 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.911099    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-99wrt\""
Sep 18 20:05:02.911468 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.911451    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:02.911545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.911461    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
Sep 18 20:05:02.918315 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:02.918291    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"]
Sep 18 20:05:03.003929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.003891    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-tmp\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.003929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.003926    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.003944    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqt92\" (UniqueName: \"kubernetes.io/projected/8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d-kube-api-access-xqt92\") pod \"network-check-source-7cbc45fddc-xq5cn\" (UID: \"8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.003966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtjxh\" (UniqueName: \"kubernetes.io/projected/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-kube-api-access-dtjxh\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.004141    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:03.504117332 +0000 UTC m=+247.633419276 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : configmap references non-existent config key: service-ca.crt
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-default-certificate\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.004231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004214    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-snapshots\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e39a38af-479b-4c64-91d2-70f12b9525b4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004281    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-smcbs\" (UniqueName: \"kubernetes.io/projected/e39a38af-479b-4c64-91d2-70f12b9525b4-kube-api-access-smcbs\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004327    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sjsf\" (UniqueName: \"kubernetes.io/projected/78a62a93-f02f-4931-ac53-4e6f63f0d96d-kube-api-access-4sjsf\") pod \"volume-data-source-validator-5c4db8bdf9-rnwzn\" (UID: \"78a62a93-f02f-4931-ac53-4e6f63f0d96d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-tmp\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004364    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8rnxd\" (UniqueName: \"kubernetes.io/projected/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-kube-api-access-8rnxd\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.004540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004530    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-stats-auth\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004573    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004624    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-serving-cert\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e39a38af-479b-4c64-91d2-70f12b9525b4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004682    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.004782    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 20:05:03.004825 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.004828    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:03.504811342 +0000 UTC m=+247.634113272 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : secret "router-metrics-certs-default" not found
Sep 18 20:05:03.005160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.004992    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-snapshots\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.005160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.005040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-trusted-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.005160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.005122    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-service-ca-bundle\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.005376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.005339    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/e39a38af-479b-4c64-91d2-70f12b9525b4-config\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.006743 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.006719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/e39a38af-479b-4c64-91d2-70f12b9525b4-serving-cert\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.007030 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.007006    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-serving-cert\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.007153 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.007134    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-default-certificate\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.007194 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.007140    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-stats-auth\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.012998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.012926    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqt92\" (UniqueName: \"kubernetes.io/projected/8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d-kube-api-access-xqt92\") pod \"network-check-source-7cbc45fddc-xq5cn\" (UID: \"8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d\") " pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"
Sep 18 20:05:03.012998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.012990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-smcbs\" (UniqueName: \"kubernetes.io/projected/e39a38af-479b-4c64-91d2-70f12b9525b4-kube-api-access-smcbs\") pod \"kube-storage-version-migrator-operator-55889f7b47-k6krk\" (UID: \"e39a38af-479b-4c64-91d2-70f12b9525b4\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.013227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.013160    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtjxh\" (UniqueName: \"kubernetes.io/projected/5b564ccd-d8f3-4d3a-91f6-bebb2043fccb-kube-api-access-dtjxh\") pod \"insights-operator-6f9c644dd9-2pxgc\" (UID: \"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb\") " pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.013682 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.013665    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rnxd\" (UniqueName: \"kubernetes.io/projected/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-kube-api-access-8rnxd\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.105999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.105888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sjsf\" (UniqueName: \"kubernetes.io/projected/78a62a93-f02f-4931-ac53-4e6f63f0d96d-kube-api-access-4sjsf\") pod \"volume-data-source-validator-5c4db8bdf9-rnwzn\" (UID: \"78a62a93-f02f-4931-ac53-4e6f63f0d96d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"
Sep 18 20:05:03.114862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.114839    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sjsf\" (UniqueName: \"kubernetes.io/projected/78a62a93-f02f-4931-ac53-4e6f63f0d96d-kube-api-access-4sjsf\") pod \"volume-data-source-validator-5c4db8bdf9-rnwzn\" (UID: \"78a62a93-f02f-4931-ac53-4e6f63f0d96d\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"
Sep 18 20:05:03.118647 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.118619    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"
Sep 18 20:05:03.129504 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.129472    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"
Sep 18 20:05:03.136676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.136648    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc"
Sep 18 20:05:03.217259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.217131    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"
Sep 18 20:05:03.396180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.396074    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn"]
Sep 18 20:05:03.400292 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:03.400262    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d4fd62b_fc16_4f13_bfc1_bcdb66a6a35d.slice/crio-6d6f210eb09cdd9a171e1d1e088f7b0cee42c2814f9c2681992c2226d5f4d41d WatchSource:0}: Error finding container 6d6f210eb09cdd9a171e1d1e088f7b0cee42c2814f9c2681992c2226d5f4d41d: Status 404 returned error can't find the container with id 6d6f210eb09cdd9a171e1d1e088f7b0cee42c2814f9c2681992c2226d5f4d41d
Sep 18 20:05:03.417669 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.417644    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk"]
Sep 18 20:05:03.423527 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.423501    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-6f9c644dd9-2pxgc"]
Sep 18 20:05:03.427170 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:03.427141    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode39a38af_479b_4c64_91d2_70f12b9525b4.slice/crio-69b84eb5e61507300318049ed64ba35c067940d03d4643cfcb9d18f17372df8c WatchSource:0}: Error finding container 69b84eb5e61507300318049ed64ba35c067940d03d4643cfcb9d18f17372df8c: Status 404 returned error can't find the container with id 69b84eb5e61507300318049ed64ba35c067940d03d4643cfcb9d18f17372df8c
Sep 18 20:05:03.427441 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:03.427418    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b564ccd_d8f3_4d3a_91f6_bebb2043fccb.slice/crio-472c18ea1da33ea986e9b7147d837fe89f7cd321c8f56eeb03f11171ec6ba1d7 WatchSource:0}: Error finding container 472c18ea1da33ea986e9b7147d837fe89f7cd321c8f56eeb03f11171ec6ba1d7: Status 404 returned error can't find the container with id 472c18ea1da33ea986e9b7147d837fe89f7cd321c8f56eeb03f11171ec6ba1d7
Sep 18 20:05:03.491382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.491324    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn"]
Sep 18 20:05:03.495720 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:03.495691    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78a62a93_f02f_4931_ac53_4e6f63f0d96d.slice/crio-209f253342b397981ffa2dc040d545d5bf02ef149bd6bf8922e908bc7b6d3c1c WatchSource:0}: Error finding container 209f253342b397981ffa2dc040d545d5bf02ef149bd6bf8922e908bc7b6d3c1c: Status 404 returned error can't find the container with id 209f253342b397981ffa2dc040d545d5bf02ef149bd6bf8922e908bc7b6d3c1c
Sep 18 20:05:03.508946 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.508911    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.509113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.508953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:03.509185 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.509107    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 20:05:03.509232 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.509179    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:04.509157323 +0000 UTC m=+248.638459258 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : configmap references non-existent config key: service-ca.crt
Sep 18 20:05:03.509232 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:03.509208    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:04.509197847 +0000 UTC m=+248.638499781 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : secret "router-metrics-certs-default" not found
Sep 18 20:05:03.794354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.794318    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn" event={"ID":"78a62a93-f02f-4931-ac53-4e6f63f0d96d","Type":"ContainerStarted","Data":"209f253342b397981ffa2dc040d545d5bf02ef149bd6bf8922e908bc7b6d3c1c"}
Sep 18 20:05:03.795539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.795509    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk" event={"ID":"e39a38af-479b-4c64-91d2-70f12b9525b4","Type":"ContainerStarted","Data":"69b84eb5e61507300318049ed64ba35c067940d03d4643cfcb9d18f17372df8c"}
Sep 18 20:05:03.796982 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.796954    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn" event={"ID":"8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d","Type":"ContainerStarted","Data":"8ee3e35b2c1f48a04f523eb15c46d920f91c4a9c791d81d7aa796f6bb32f5c8e"}
Sep 18 20:05:03.797145 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.796999    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn" event={"ID":"8d4fd62b-fc16-4f13-bfc1-bcdb66a6a35d","Type":"ContainerStarted","Data":"6d6f210eb09cdd9a171e1d1e088f7b0cee42c2814f9c2681992c2226d5f4d41d"}
Sep 18 20:05:03.798131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.798108    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc" event={"ID":"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb","Type":"ContainerStarted","Data":"472c18ea1da33ea986e9b7147d837fe89f7cd321c8f56eeb03f11171ec6ba1d7"}
Sep 18 20:05:03.812820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:03.812773    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-7cbc45fddc-xq5cn" podStartSLOduration=1.812759719 podStartE2EDuration="1.812759719s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:03.812199384 +0000 UTC m=+247.941501334" watchObservedRunningTime="2026-09-18 20:05:03.812759719 +0000 UTC m=+247.942061670"
Sep 18 20:05:04.518666 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:04.517699    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:04.518666 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:04.517751    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:04.518666 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:04.518092    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:06.518070291 +0000 UTC m=+250.647372236 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : configmap references non-existent config key: service-ca.crt
Sep 18 20:05:04.518666 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:04.518573    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 20:05:04.518666 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:04.518625    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:06.518609533 +0000 UTC m=+250.647911468 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : secret "router-metrics-certs-default" not found
Sep 18 20:05:06.534877 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.534824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:06.535307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.534886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:06.535307 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:06.535002    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 20:05:06.535307 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:06.535084    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.535066041 +0000 UTC m=+254.664367983 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : configmap references non-existent config key: service-ca.crt
Sep 18 20:05:06.535307 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:06.535107    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.535100356 +0000 UTC m=+254.664402285 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : secret "router-metrics-certs-default" not found
Sep 18 20:05:06.809050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.808951    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn" event={"ID":"78a62a93-f02f-4931-ac53-4e6f63f0d96d","Type":"ContainerStarted","Data":"c47755892cbe750c47aac15f07abce98166f7dcafa9f88000183cde45d4949c0"}
Sep 18 20:05:06.810642 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.810617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk" event={"ID":"e39a38af-479b-4c64-91d2-70f12b9525b4","Type":"ContainerStarted","Data":"01b005b822b628d48bf6b1d5485dd4e5c9d6ae7deccf6714b7d1c5ede65605b8"}
Sep 18 20:05:06.812096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.812076    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc" event={"ID":"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb","Type":"ContainerStarted","Data":"feb1be49bad7bf440200a19811bad1e197cfce76551a03e63e307096d4559b66"}
Sep 18 20:05:06.823228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.823181    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5c4db8bdf9-rnwzn" podStartSLOduration=2.378110799 podStartE2EDuration="4.823165002s" totalImagesPullingTime="2.445054203s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:06.822754788 +0000 UTC m=+250.952056761" watchObservedRunningTime="2026-09-18 20:05:06.823165002 +0000 UTC m=+250.952466952"
Sep 18 20:05:06.837439 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.837371    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk" podStartSLOduration=2.3194044480000002 podStartE2EDuration="4.837349194s" totalImagesPullingTime="2.517944746s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:06.836735998 +0000 UTC m=+250.966037950" watchObservedRunningTime="2026-09-18 20:05:06.837349194 +0000 UTC m=+250.966651146"
Sep 18 20:05:06.854175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:06.854108    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc" podStartSLOduration=2.33552139 podStartE2EDuration="4.854084486s" totalImagesPullingTime="2.518563096s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:06.852846146 +0000 UTC m=+250.982148098" watchObservedRunningTime="2026-09-18 20:05:06.854084486 +0000 UTC m=+250.983386439"
Sep 18 20:05:07.045153 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.045112    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"]
Sep 18 20:05:07.047738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.047713    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"
Sep 18 20:05:07.050180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.050147    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 18 20:05:07.050316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.050212    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 18 20:05:07.050316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.050214    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-kklnc\""
Sep 18 20:05:07.058702 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.058671    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"]
Sep 18 20:05:07.140787 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.140681    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hcv4h\" (UniqueName: \"kubernetes.io/projected/0c470b42-45f0-464f-8714-de32ee575948-kube-api-access-hcv4h\") pod \"migrator-76794cbd6c-hczmk\" (UID: \"0c470b42-45f0-464f-8714-de32ee575948\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"
Sep 18 20:05:07.241892 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.241820    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hcv4h\" (UniqueName: \"kubernetes.io/projected/0c470b42-45f0-464f-8714-de32ee575948-kube-api-access-hcv4h\") pod \"migrator-76794cbd6c-hczmk\" (UID: \"0c470b42-45f0-464f-8714-de32ee575948\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"
Sep 18 20:05:07.247214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.247177    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-cctw5"]
Sep 18 20:05:07.250221 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.250197    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.252443 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.252420    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 18 20:05:07.252571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.252445    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 18 20:05:07.252571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.252428    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-z7xss\""
Sep 18 20:05:07.258023 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.258002    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hcv4h\" (UniqueName: \"kubernetes.io/projected/0c470b42-45f0-464f-8714-de32ee575948-kube-api-access-hcv4h\") pod \"migrator-76794cbd6c-hczmk\" (UID: \"0c470b42-45f0-464f-8714-de32ee575948\") " pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"
Sep 18 20:05:07.263616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.263592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cctw5"]
Sep 18 20:05:07.342367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.342326    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-data-volume\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.342553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.342388    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2dbs\" (UniqueName: \"kubernetes.io/projected/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-api-access-q2dbs\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.342553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.342485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.342553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.342513    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.342553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.342549    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-crio-socket\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.359422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.359379    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"
Sep 18 20:05:07.443312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443263    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443312    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443351    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-crio-socket\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443375    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-data-volume\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.443432    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443477    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-crio-socket\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.443511    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:07.943490923 +0000 UTC m=+252.072792879 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.443574 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.443512    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.443839 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.443618    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:07.943598474 +0000 UTC m=+252.072900425 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.443839 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443550    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q2dbs\" (UniqueName: \"kubernetes.io/projected/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-api-access-q2dbs\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.443994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.443974    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-data-volume\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.451827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.451791    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2dbs\" (UniqueName: \"kubernetes.io/projected/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-api-access-q2dbs\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.630077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.630022    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk"]
Sep 18 20:05:07.633611 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:07.633579    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c470b42_45f0_464f_8714_de32ee575948.slice/crio-6f5aa0d3ee49e6cfba60a127b1f3280452ecd07e732d185b36f4a2eee7c559b6 WatchSource:0}: Error finding container 6f5aa0d3ee49e6cfba60a127b1f3280452ecd07e732d185b36f4a2eee7c559b6: Status 404 returned error can't find the container with id 6f5aa0d3ee49e6cfba60a127b1f3280452ecd07e732d185b36f4a2eee7c559b6
Sep 18 20:05:07.816876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.816827    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk" event={"ID":"0c470b42-45f0-464f-8714-de32ee575948","Type":"ContainerStarted","Data":"6f5aa0d3ee49e6cfba60a127b1f3280452ecd07e732d185b36f4a2eee7c559b6"}
Sep 18 20:05:07.948964 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.948913    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.948964 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:07.948966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:07.949254 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.949034    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.949254 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.949093    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:08.949079902 +0000 UTC m=+253.078381831 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:07.949254 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.949102    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:07.949254 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:07.949164    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:08.949147719 +0000 UTC m=+253.078449663 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:08.959141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:08.959097    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:08.959141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:08.959150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:08.959706 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:08.959281    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:08.959706 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:08.959282    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:08.959706 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:08.959357    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.959335244 +0000 UTC m=+255.088637175 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:08.959706 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:08.959423    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:10.959409205 +0000 UTC m=+255.088711135 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:09.816425 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:09.816398    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-67q5n_b69fb34b-9443-4cea-9b7c-19ba8258730a/dns-node-resolver/0.log"
Sep 18 20:05:09.825664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:09.825626    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk" event={"ID":"0c470b42-45f0-464f-8714-de32ee575948","Type":"ContainerStarted","Data":"9b7202c0de3e3bc09a211d389608759685794198ca7560235810c29b5310aeb9"}
Sep 18 20:05:09.825844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:09.825680    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk" event={"ID":"0c470b42-45f0-464f-8714-de32ee575948","Type":"ContainerStarted","Data":"fc4ca3c98b10d98bdef1c058b876a6d19edac6afe28142e156cc39c087f43d06"}
Sep 18 20:05:09.840712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:09.840658    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-76794cbd6c-hczmk" podStartSLOduration=1.417002933 podStartE2EDuration="2.840641225s" totalImagesPullingTime="1.423638292s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:09.839024124 +0000 UTC m=+253.968326074" watchObservedRunningTime="2026-09-18 20:05:09.840641225 +0000 UTC m=+253.969943176"
Sep 18 20:05:09.868190 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:09.868149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:05:09.872169 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:09.868775    2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
Sep 18 20:05:09.872169 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:09.868879    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs podName:d7a6ca69-b215-437a-9191-d46540f76181 nodeName:}" failed. No retries permitted until 2026-09-18 20:07:11.868857138 +0000 UTC m=+375.998159083 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs") pod "network-metrics-daemon-4jr7x" (UID: "d7a6ca69-b215-437a-9191-d46540f76181") : secret "metrics-daemon-secret" not found
Sep 18 20:05:10.574181 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:10.574134    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:10.574181 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:10.574180    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:10.574643 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.574289    2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
Sep 18 20:05:10.574643 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.574292    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:18.574278598 +0000 UTC m=+262.703580528 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : configmap references non-existent config key: service-ca.crt
Sep 18 20:05:10.574643 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.574349    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs podName:4b3df289-28e4-4f05-a9be-fc6f09f22ef6 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:18.574335498 +0000 UTC m=+262.703637462 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs") pod "router-default-94896f94d-jhtw4" (UID: "4b3df289-28e4-4f05-a9be-fc6f09f22ef6") : secret "router-metrics-certs-default" not found
Sep 18 20:05:10.976910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:10.976870    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:10.976910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:10.976914    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:10.977139 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.977015    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:10.977139 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.977024    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:10.977139 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.977089    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:14.977070007 +0000 UTC m=+259.106371953 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:10.977139 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:10.977104    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:14.977097527 +0000 UTC m=+259.106399456 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:11.017051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:11.017023    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7h7pk_5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755/node-ca/0.log"
Sep 18 20:05:12.017479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:12.017450    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrnwv_a6e4fdd6-d434-4c3f-af0b-0f7531647a85/serve-healthcheck-canary/0.log"
Sep 18 20:05:12.220068 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:12.220034    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-hczmk_0c470b42-45f0-464f-8714-de32ee575948/migrator/0.log"
Sep 18 20:05:12.417141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:12.417049    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-hczmk_0c470b42-45f0-464f-8714-de32ee575948/graceful-termination/0.log"
Sep 18 20:05:12.617834 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:12.617801    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-k6krk_e39a38af-479b-4c64-91d2-70f12b9525b4/kube-storage-version-migrator-operator/0.log"
Sep 18 20:05:15.011689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:15.011602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:15.011689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:15.011693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:15.012630 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:15.011843    2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 18 20:05:15.012630 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:15.011870    2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:15.012630 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:15.011955    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:23.011925066 +0000 UTC m=+267.141227013 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : configmap "kube-rbac-proxy" not found
Sep 18 20:05:15.012630 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:15.011982    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls podName:9e3eee4c-92e3-4095-b0e6-d00eb6893fa3 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:23.0119709 +0000 UTC m=+267.141272832 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls") pod "insights-runtime-extractor-cctw5" (UID: "9e3eee4c-92e3-4095-b0e6-d00eb6893fa3") : secret "insights-runtime-extractor-tls" not found
Sep 18 20:05:18.645398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:18.645354    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:18.645398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:18.645406    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:18.646076 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:18.646037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-service-ca-bundle\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:18.647933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:18.647907    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4b3df289-28e4-4f05-a9be-fc6f09f22ef6-metrics-certs\") pod \"router-default-94896f94d-jhtw4\" (UID: \"4b3df289-28e4-4f05-a9be-fc6f09f22ef6\") " pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:18.742140 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:18.742094    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:19.013405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:19.013354    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-94896f94d-jhtw4"]
Sep 18 20:05:19.017165 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:19.017127    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3df289_28e4_4f05_a9be_fc6f09f22ef6.slice/crio-a38aaa0dda612a4edbe1b95dd8140799d967e6f0469f22e292cc46374b3ec8b6 WatchSource:0}: Error finding container a38aaa0dda612a4edbe1b95dd8140799d967e6f0469f22e292cc46374b3ec8b6: Status 404 returned error can't find the container with id a38aaa0dda612a4edbe1b95dd8140799d967e6f0469f22e292cc46374b3ec8b6
Sep 18 20:05:19.861272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:19.861236    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-94896f94d-jhtw4" event={"ID":"4b3df289-28e4-4f05-a9be-fc6f09f22ef6","Type":"ContainerStarted","Data":"58a5ee34fd966f7f523699c37d5a1386a3581c944951a9069db016d1a2a7d045"}
Sep 18 20:05:19.861651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:19.861286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-94896f94d-jhtw4" event={"ID":"4b3df289-28e4-4f05-a9be-fc6f09f22ef6","Type":"ContainerStarted","Data":"a38aaa0dda612a4edbe1b95dd8140799d967e6f0469f22e292cc46374b3ec8b6"}
Sep 18 20:05:19.879258 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:19.879201    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-94896f94d-jhtw4" podStartSLOduration=17.879179153 podStartE2EDuration="17.879179153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:02 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:19.878768448 +0000 UTC m=+264.008070427" watchObservedRunningTime="2026-09-18 20:05:19.879179153 +0000 UTC m=+264.008481105"
Sep 18 20:05:20.742654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:20.742611    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:20.745285 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:20.745259    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:20.864424 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:20.864391    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:20.865721 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:20.865703    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-94896f94d-jhtw4"
Sep 18 20:05:23.084876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.084834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:23.084876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.084878    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:23.085391 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.085374    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:23.087279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.087250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9e3eee4c-92e3-4095-b0e6-d00eb6893fa3-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-cctw5\" (UID: \"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3\") " pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:23.160128 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.160052    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-cctw5"
Sep 18 20:05:23.433289 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.433202    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-cctw5"]
Sep 18 20:05:23.436769 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:23.436727    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e3eee4c_92e3_4095_b0e6_d00eb6893fa3.slice/crio-ec12879236096147b4ec41de31567b364d81ecdcf55a090d20bf2a2a40e3afcc WatchSource:0}: Error finding container ec12879236096147b4ec41de31567b364d81ecdcf55a090d20bf2a2a40e3afcc: Status 404 returned error can't find the container with id ec12879236096147b4ec41de31567b364d81ecdcf55a090d20bf2a2a40e3afcc
Sep 18 20:05:23.876652 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.876617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cctw5" event={"ID":"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3","Type":"ContainerStarted","Data":"79e7a05207b0812793e20d11ebb6af1e5cc3ad3a062fb6b78b618fcd4d67b52e"}
Sep 18 20:05:23.876824 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:23.876673    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cctw5" event={"ID":"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3","Type":"ContainerStarted","Data":"ec12879236096147b4ec41de31567b364d81ecdcf55a090d20bf2a2a40e3afcc"}
Sep 18 20:05:24.881225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:24.881191    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cctw5" event={"ID":"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3","Type":"ContainerStarted","Data":"ed588f3a7ba8ee256c779630db9774810830d03615abb7ea676fd1782d03e204"}
Sep 18 20:05:25.886284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:25.886195    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-cctw5" event={"ID":"9e3eee4c-92e3-4095-b0e6-d00eb6893fa3","Type":"ContainerStarted","Data":"4169b09d56697b4f570207b161deb5a8166b3a2cbd1b0215178aba0c1004dc95"}
Sep 18 20:05:25.903581 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:25.903531    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-cctw5" podStartSLOduration=16.828688171 podStartE2EDuration="18.90351234s" totalImagesPullingTime="2.074824169s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:05:07 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:25.902710765 +0000 UTC m=+270.032012719" watchObservedRunningTime="2026-09-18 20:05:25.90351234 +0000 UTC m=+270.032814290"
Sep 18 20:05:31.470977 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.470940    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:05:31.473323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.473305    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.475752 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.475727    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 20:05:31.475876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.475728    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 20:05:31.477591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477134    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 20:05:31.477591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477167    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 20:05:31.477591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477228    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:05:31.477591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477171    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j65gf\""
Sep 18 20:05:31.477591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477521    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:05:31.477924 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.477850    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 20:05:31.484570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.484303    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:05:31.556607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.556607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556615    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.556825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556691    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.556825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556740    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44r6j\" (UniqueName: \"kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.556825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556759    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.556825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.556814    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.657997 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.657960    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.657997 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658006    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.658273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658076    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.658273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658119    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44r6j\" (UniqueName: \"kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.658273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658147    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.658427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658372    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.658894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658873    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.659011 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658917    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.659011 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.658991    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.660554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.660533    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.660641 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.660541    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.666284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.666258    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44r6j\" (UniqueName: \"kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j\") pod \"console-7585545484-nddg2\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") " pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:31.788149 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:31.788030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:32.056037 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:32.055952    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:05:32.059030 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:32.058999    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod129315b1_277d_48a3_8648_fcb2d1be3395.slice/crio-c922dc66e1b72d341be1102f6a4be2d25606b016bd0b009cf7c53f27926d26b8 WatchSource:0}: Error finding container c922dc66e1b72d341be1102f6a4be2d25606b016bd0b009cf7c53f27926d26b8: Status 404 returned error can't find the container with id c922dc66e1b72d341be1102f6a4be2d25606b016bd0b009cf7c53f27926d26b8
Sep 18 20:05:32.913614 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:32.913429    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7585545484-nddg2" event={"ID":"129315b1-277d-48a3-8648-fcb2d1be3395","Type":"ContainerStarted","Data":"c922dc66e1b72d341be1102f6a4be2d25606b016bd0b009cf7c53f27926d26b8"}
Sep 18 20:05:36.928856 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:36.928819    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7585545484-nddg2" event={"ID":"129315b1-277d-48a3-8648-fcb2d1be3395","Type":"ContainerStarted","Data":"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"}
Sep 18 20:05:36.946508 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:36.946426    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7585545484-nddg2" podStartSLOduration=2.007666751 podStartE2EDuration="5.946410332s" totalImagesPullingTime="3.938743581s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:36.945896412 +0000 UTC m=+281.075198364" watchObservedRunningTime="2026-09-18 20:05:36.946410332 +0000 UTC m=+281.075712338"
Sep 18 20:05:37.584829 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:37.584773    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1"
Sep 18 20:05:37.584829 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:37.584773    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-qvdjd" podUID="bd5be920-996d-492f-8de2-83ab8c7b1d30"
Sep 18 20:05:37.584829 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:37.584783    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8" podUID="c397f480-13a7-4cd9-8dd5-690d317bfb4d"
Sep 18 20:05:37.931670 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:37.931585    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:05:37.931670 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:37.931615    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:05:37.932159 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:37.931777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:41.235985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.235936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:41.235985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.235992    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:05:41.236457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.236019    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:05:41.238525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.238496    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bd5be920-996d-492f-8de2-83ab8c7b1d30-metrics-tls\") pod \"dns-default-qvdjd\" (UID: \"bd5be920-996d-492f-8de2-83ab8c7b1d30\") " pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:41.238525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.238513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"image-registry-65c945669c-gwd5f\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") " pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:05:41.238692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.238601    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/c397f480-13a7-4cd9-8dd5-690d317bfb4d-networking-console-plugin-cert\") pod \"networking-console-plugin-6446c8d75d-9csd8\" (UID: \"c397f480-13a7-4cd9-8dd5-690d317bfb4d\") " pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:05:41.535354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.535262    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-vl2fh\""
Sep 18 20:05:41.535354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.535279    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-t5jb8\""
Sep 18 20:05:41.535354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.535292    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-dw2ht\""
Sep 18 20:05:41.543259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.543227    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"
Sep 18 20:05:41.543259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.543250    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:41.543480 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.543269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:05:41.757922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.757886    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"]
Sep 18 20:05:41.774966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.774936    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"]
Sep 18 20:05:41.775129 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.775037    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:41.777552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.777532    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
Sep 18 20:05:41.778355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.778334    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
Sep 18 20:05:41.778487 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.778376    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 18 20:05:41.778487 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.778344    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 18 20:05:41.778487 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.778400    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 18 20:05:41.778487 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.778415    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-cwdcq\""
Sep 18 20:05:41.788971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.788920    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:41.789099 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.788997    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:05:41.790523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.790491    2566 patch_prober.go:28] interesting pod/console-7585545484-nddg2 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body=
Sep 18 20:05:41.790632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.790577    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7585545484-nddg2" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused"
Sep 18 20:05:41.861841 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.861757    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-6446c8d75d-9csd8"]
Sep 18 20:05:41.864565 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:41.864535    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc397f480_13a7_4cd9_8dd5_690d317bfb4d.slice/crio-cf3a7a89b7ae68686f08a0fec1e3e812d18f716ab0361954f053fbb234983b93 WatchSource:0}: Error finding container cf3a7a89b7ae68686f08a0fec1e3e812d18f716ab0361954f053fbb234983b93: Status 404 returned error can't find the container with id cf3a7a89b7ae68686f08a0fec1e3e812d18f716ab0361954f053fbb234983b93
Sep 18 20:05:41.887617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.887437    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qvdjd"]
Sep 18 20:05:41.890067 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:41.890022    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd5be920_996d_492f_8de2_83ab8c7b1d30.slice/crio-e085233e320786397a916da4dcbe1ceb10da27725478ad2990f23f7275c707f1 WatchSource:0}: Error finding container e085233e320786397a916da4dcbe1ceb10da27725478ad2990f23f7275c707f1: Status 404 returned error can't find the container with id e085233e320786397a916da4dcbe1ceb10da27725478ad2990f23f7275c707f1
Sep 18 20:05:41.913004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.912973    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:05:41.916150 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:41.916107    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5ab2815_9837_437a_bae0_c11f02acaac1.slice/crio-9a7262e16393973d9ddac7b8dbf3b377a6dd5a64bce943e0f91a562851575224 WatchSource:0}: Error finding container 9a7262e16393973d9ddac7b8dbf3b377a6dd5a64bce943e0f91a562851575224: Status 404 returned error can't find the container with id 9a7262e16393973d9ddac7b8dbf3b377a6dd5a64bce943e0f91a562851575224
Sep 18 20:05:41.944261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.944225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4wvmj\" (UniqueName: \"kubernetes.io/projected/4887d16f-9391-419b-9a85-1560af904530-kube-api-access-4wvmj\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:41.944434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.944341    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:41.944434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.944371    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4887d16f-9391-419b-9a85-1560af904530-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:41.944434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.944394    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:41.951718 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.951541    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qvdjd" event={"ID":"bd5be920-996d-492f-8de2-83ab8c7b1d30","Type":"ContainerStarted","Data":"e085233e320786397a916da4dcbe1ceb10da27725478ad2990f23f7275c707f1"}
Sep 18 20:05:41.952750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.952726    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8" event={"ID":"c397f480-13a7-4cd9-8dd5-690d317bfb4d","Type":"ContainerStarted","Data":"cf3a7a89b7ae68686f08a0fec1e3e812d18f716ab0361954f053fbb234983b93"}
Sep 18 20:05:41.953921 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:41.953901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" event={"ID":"a5ab2815-9837-437a-bae0-c11f02acaac1","Type":"ContainerStarted","Data":"9a7262e16393973d9ddac7b8dbf3b377a6dd5a64bce943e0f91a562851575224"}
Sep 18 20:05:42.045842 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.045760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4wvmj\" (UniqueName: \"kubernetes.io/projected/4887d16f-9391-419b-9a85-1560af904530-kube-api-access-4wvmj\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.046313 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.045849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.046313 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.045879    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4887d16f-9391-419b-9a85-1560af904530-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.046313 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.045906    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.046313 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:42.046153    2566 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found
Sep 18 20:05:42.046313 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:42.046222    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls podName:4887d16f-9391-419b-9a85-1560af904530 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:42.54620144 +0000 UTC m=+286.675503376 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls") pod "prometheus-operator-7bd5d56d6-cckpb" (UID: "4887d16f-9391-419b-9a85-1560af904530") : secret "prometheus-operator-tls" not found
Sep 18 20:05:42.046686 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.046655    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4887d16f-9391-419b-9a85-1560af904530-metrics-client-ca\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.048288 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.048267    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.053695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.053668    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4wvmj\" (UniqueName: \"kubernetes.io/projected/4887d16f-9391-419b-9a85-1560af904530-kube-api-access-4wvmj\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.551919 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.551819    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.554812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.554774    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4887d16f-9391-419b-9a85-1560af904530-prometheus-operator-tls\") pod \"prometheus-operator-7bd5d56d6-cckpb\" (UID: \"4887d16f-9391-419b-9a85-1560af904530\") " pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.685216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.685174    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"
Sep 18 20:05:42.963462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.963418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" event={"ID":"a5ab2815-9837-437a-bae0-c11f02acaac1","Type":"ContainerStarted","Data":"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"}
Sep 18 20:05:42.963663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.963501    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:05:42.972127 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.972069    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb"]
Sep 18 20:05:42.986138 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:42.986075    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" podStartSLOduration=284.986040573 podStartE2EDuration="4m44.986040573s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:42.984885281 +0000 UTC m=+287.114187233" watchObservedRunningTime="2026-09-18 20:05:42.986040573 +0000 UTC m=+287.115342524"
Sep 18 20:05:43.065976 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:43.065930    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4887d16f_9391_419b_9a85_1560af904530.slice/crio-c6fec81d3ebe1fcabb5ea8ee079d09df397419ae372a72dacc8e9d2cf4646c5f WatchSource:0}: Error finding container c6fec81d3ebe1fcabb5ea8ee079d09df397419ae372a72dacc8e9d2cf4646c5f: Status 404 returned error can't find the container with id c6fec81d3ebe1fcabb5ea8ee079d09df397419ae372a72dacc8e9d2cf4646c5f
Sep 18 20:05:43.970525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.970486    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qvdjd" event={"ID":"bd5be920-996d-492f-8de2-83ab8c7b1d30","Type":"ContainerStarted","Data":"beccdcd5f02a55be20ded40e550496ceddd2adf0b303c6b952086c80b2b3ec14"}
Sep 18 20:05:43.971022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.970554    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qvdjd" event={"ID":"bd5be920-996d-492f-8de2-83ab8c7b1d30","Type":"ContainerStarted","Data":"80515cbda6ab6c4a4685e751643a5b6eb7011984c0071bc8a7a477d6f7237447"}
Sep 18 20:05:43.971022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.970588    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:43.972050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.971991    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb" event={"ID":"4887d16f-9391-419b-9a85-1560af904530","Type":"ContainerStarted","Data":"c6fec81d3ebe1fcabb5ea8ee079d09df397419ae372a72dacc8e9d2cf4646c5f"}
Sep 18 20:05:43.973829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.973798    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8" event={"ID":"c397f480-13a7-4cd9-8dd5-690d317bfb4d","Type":"ContainerStarted","Data":"3b6ed1500b8214bfbb582a745ce6297fbb96f3ba72797c60bf5c31462ac43380"}
Sep 18 20:05:43.987803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:43.987752    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qvdjd" podStartSLOduration=251.188330288 podStartE2EDuration="4m12.987735695s" totalImagesPullingTime="1.799405407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:43.987605691 +0000 UTC m=+288.116907642" watchObservedRunningTime="2026-09-18 20:05:43.987735695 +0000 UTC m=+288.117037642"
Sep 18 20:05:44.002322 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:44.002252    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-6446c8d75d-9csd8" podStartSLOduration=278.771757874 podStartE2EDuration="4m40.002235207s" totalImagesPullingTime="1.230477333s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:01:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:44.001395068 +0000 UTC m=+288.130697019" watchObservedRunningTime="2026-09-18 20:05:44.002235207 +0000 UTC m=+288.131537158"
Sep 18 20:05:44.979691 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:44.979656    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb" event={"ID":"4887d16f-9391-419b-9a85-1560af904530","Type":"ContainerStarted","Data":"0d2aaaaa2ee3fb44a953611517daf3c58093e7e10df197eca590368e138603aa"}
Sep 18 20:05:44.980128 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:44.979717    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb" event={"ID":"4887d16f-9391-419b-9a85-1560af904530","Type":"ContainerStarted","Data":"b17b0c72cdd557140b3d77a850d89bd66c18922ff0998f7608146988b7a8b7c8"}
Sep 18 20:05:44.996019 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:44.995960    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7bd5d56d6-cckpb" podStartSLOduration=2.52895766 podStartE2EDuration="3.995937883s" totalImagesPullingTime="1.466980223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:44.994929657 +0000 UTC m=+289.124231608" watchObservedRunningTime="2026-09-18 20:05:44.995937883 +0000 UTC m=+289.125239833"
Sep 18 20:05:47.150338 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.150299    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"]
Sep 18 20:05:47.152487 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.152470    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.155143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.155112    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"]
Sep 18 20:05:47.155286 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.155171    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
Sep 18 20:05:47.155286 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.155244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-ngjts\""
Sep 18 20:05:47.155286 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.155244    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
Sep 18 20:05:47.157636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.157617    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.160537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.160274    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
Sep 18 20:05:47.160628 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.160571    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
Sep 18 20:05:47.160970 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.160953    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9zv9z\""
Sep 18 20:05:47.161751 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.161734    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
Sep 18 20:05:47.169113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.169024    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"]
Sep 18 20:05:47.174823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.174793    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"]
Sep 18 20:05:47.175905 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.175883    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-m5fhs"]
Sep 18 20:05:47.178227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.178208    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.180758 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.180739    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 18 20:05:47.180847 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.180821    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 18 20:05:47.181014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.180994    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-bvmbl\""
Sep 18 20:05:47.181169 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.181031    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 18 20:05:47.289031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.288987    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-textfile\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289045    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289102    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g8f2l\" (UniqueName: \"kubernetes.io/projected/659002d6-e06b-4bce-9329-61b69116bb31-kube-api-access-g8f2l\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dc86046-9015-44f6-b8fc-2fad8d868f13-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289192    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289214    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z2dj\" (UniqueName: \"kubernetes.io/projected/6dc86046-9015-44f6-b8fc-2fad8d868f13-kube-api-access-9z2dj\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.289240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289236    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-tls\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289290    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-sys\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289342    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289361    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289375    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-root\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289418    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/659002d6-e06b-4bce-9329-61b69116bb31-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jttpp\" (UniqueName: \"kubernetes.io/projected/c2993b00-deeb-400f-9c59-306ba1adc201-kube-api-access-jttpp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289467    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.289693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289485    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-metrics-client-ca\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.289693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289512    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289542    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.289693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.289564    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-wtmp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.389987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.389938    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g8f2l\" (UniqueName: \"kubernetes.io/projected/659002d6-e06b-4bce-9329-61b69116bb31-kube-api-access-g8f2l\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.389985    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dc86046-9015-44f6-b8fc-2fad8d868f13-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390024    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390048    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z2dj\" (UniqueName: \"kubernetes.io/projected/6dc86046-9015-44f6-b8fc-2fad8d868f13-kube-api-access-9z2dj\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390130    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-tls\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390159    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390182    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-sys\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390235    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390286    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-sys\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390284    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-root\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390356    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/659002d6-e06b-4bce-9329-61b69116bb31-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390450    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-root\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:47.390455    2566 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
Sep 18 20:05:47.390505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390484    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jttpp\" (UniqueName: \"kubernetes.io/projected/c2993b00-deeb-400f-9c59-306ba1adc201-kube-api-access-jttpp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:47.390517    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls podName:659002d6-e06b-4bce-9329-61b69116bb31 nodeName:}" failed. No retries permitted until 2026-09-18 20:05:47.890496039 +0000 UTC m=+292.019797982 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls") pod "kube-state-metrics-84c885658c-mj9nm" (UID: "659002d6-e06b-4bce-9329-61b69116bb31") : secret "kube-state-metrics-tls" not found
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390536    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390565    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-metrics-client-ca\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390675    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-wtmp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390728    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-textfile\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.390879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390748    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-accelerators-collector-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.391299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.390985    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/659002d6-e06b-4bce-9329-61b69116bb31-volume-directive-shadow\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.391299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391018    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-textfile\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.391299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.391299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391143    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-wtmp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.391505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391429    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/6dc86046-9015-44f6-b8fc-2fad8d868f13-metrics-client-ca\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.391559 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391540    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.391669 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391630    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c2993b00-deeb-400f-9c59-306ba1adc201-metrics-client-ca\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.391917 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.391883    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/659002d6-e06b-4bce-9329-61b69116bb31-metrics-client-ca\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.393006 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.392976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.393147 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.393131    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c2993b00-deeb-400f-9c59-306ba1adc201-node-exporter-tls\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.393526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.393504    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.394144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.394109    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.394224 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.394191    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/6dc86046-9015-44f6-b8fc-2fad8d868f13-openshift-state-metrics-tls\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.400847 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.400778    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jttpp\" (UniqueName: \"kubernetes.io/projected/c2993b00-deeb-400f-9c59-306ba1adc201-kube-api-access-jttpp\") pod \"node-exporter-m5fhs\" (UID: \"c2993b00-deeb-400f-9c59-306ba1adc201\") " pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.400991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.400953    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g8f2l\" (UniqueName: \"kubernetes.io/projected/659002d6-e06b-4bce-9329-61b69116bb31-kube-api-access-g8f2l\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.401137 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.401084    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z2dj\" (UniqueName: \"kubernetes.io/projected/6dc86046-9015-44f6-b8fc-2fad8d868f13-kube-api-access-9z2dj\") pod \"openshift-state-metrics-86c5bc79d6-rjc2l\" (UID: \"6dc86046-9015-44f6-b8fc-2fad8d868f13\") " pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.464162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.464107    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"
Sep 18 20:05:47.487085 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.487030    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-m5fhs"
Sep 18 20:05:47.498232 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:47.498197    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2993b00_deeb_400f_9c59_306ba1adc201.slice/crio-d7f74aef630387265cdbfdd7f778b79e48408e174ff304601d5a47c381e56556 WatchSource:0}: Error finding container d7f74aef630387265cdbfdd7f778b79e48408e174ff304601d5a47c381e56556: Status 404 returned error can't find the container with id d7f74aef630387265cdbfdd7f778b79e48408e174ff304601d5a47c381e56556
Sep 18 20:05:47.740963 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.740930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l"]
Sep 18 20:05:47.743989 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:47.743956    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6dc86046_9015_44f6_b8fc_2fad8d868f13.slice/crio-e26fbd35090a148f68fc0969d7b92172f99014e9c6fee7b0485f6afb78989718 WatchSource:0}: Error finding container e26fbd35090a148f68fc0969d7b92172f99014e9c6fee7b0485f6afb78989718: Status 404 returned error can't find the container with id e26fbd35090a148f68fc0969d7b92172f99014e9c6fee7b0485f6afb78989718
Sep 18 20:05:47.895459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.895417    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.897958 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.897929    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/659002d6-e06b-4bce-9329-61b69116bb31-kube-state-metrics-tls\") pod \"kube-state-metrics-84c885658c-mj9nm\" (UID: \"659002d6-e06b-4bce-9329-61b69116bb31\") " pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:47.996844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.996737    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l" event={"ID":"6dc86046-9015-44f6-b8fc-2fad8d868f13","Type":"ContainerStarted","Data":"56fa06e1b3cabc98faa58ab632558584107d8f2c624ae6f2c92514b4b968617d"}
Sep 18 20:05:47.997028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.996848    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l" event={"ID":"6dc86046-9015-44f6-b8fc-2fad8d868f13","Type":"ContainerStarted","Data":"dbeea08ff369b611231cfa517ab36a42b5e163260fd277259ea02db68079d034"}
Sep 18 20:05:47.997028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.996885    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l" event={"ID":"6dc86046-9015-44f6-b8fc-2fad8d868f13","Type":"ContainerStarted","Data":"e26fbd35090a148f68fc0969d7b92172f99014e9c6fee7b0485f6afb78989718"}
Sep 18 20:05:47.999022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:47.998982    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5fhs" event={"ID":"c2993b00-deeb-400f-9c59-306ba1adc201","Type":"ContainerStarted","Data":"d7f74aef630387265cdbfdd7f778b79e48408e174ff304601d5a47c381e56556"}
Sep 18 20:05:48.070446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:48.070405    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"
Sep 18 20:05:48.352114 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:48.351951    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-84c885658c-mj9nm"]
Sep 18 20:05:48.355770 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:48.355731    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod659002d6_e06b_4bce_9329_61b69116bb31.slice/crio-2a486a266dff6e34c43f7bdb7a950b63e4caf362b328546cc269591d1e7e5e3c WatchSource:0}: Error finding container 2a486a266dff6e34c43f7bdb7a950b63e4caf362b328546cc269591d1e7e5e3c: Status 404 returned error can't find the container with id 2a486a266dff6e34c43f7bdb7a950b63e4caf362b328546cc269591d1e7e5e3c
Sep 18 20:05:49.005017 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:49.004967    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5fhs" event={"ID":"c2993b00-deeb-400f-9c59-306ba1adc201","Type":"ContainerStarted","Data":"246e433a8878e7b0ddf2301855a13bef15d2a35d431ac709ac2ce02225fefbbe"}
Sep 18 20:05:49.006878 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:49.006849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm" event={"ID":"659002d6-e06b-4bce-9329-61b69116bb31","Type":"ContainerStarted","Data":"2a486a266dff6e34c43f7bdb7a950b63e4caf362b328546cc269591d1e7e5e3c"}
Sep 18 20:05:49.011047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:49.010979    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l" event={"ID":"6dc86046-9015-44f6-b8fc-2fad8d868f13","Type":"ContainerStarted","Data":"91d95975f6559fe9096e42b374bd510484f6a74f72313f43cbba60ea544b8ee5"}
Sep 18 20:05:49.039431 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:49.039335    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-86c5bc79d6-rjc2l" podStartSLOduration=0.991461791 podStartE2EDuration="2.0393077s" totalImagesPullingTime="1.047845909s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:49.03861203 +0000 UTC m=+293.167913982" watchObservedRunningTime="2026-09-18 20:05:49.0393077 +0000 UTC m=+293.168609656"
Sep 18 20:05:50.018113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.018047    2566 generic.go:356] "Generic (PLEG): container finished" podID="c2993b00-deeb-400f-9c59-306ba1adc201" containerID="246e433a8878e7b0ddf2301855a13bef15d2a35d431ac709ac2ce02225fefbbe" exitCode=0
Sep 18 20:05:50.018574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.018129    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5fhs" event={"ID":"c2993b00-deeb-400f-9c59-306ba1adc201","Type":"ContainerDied","Data":"246e433a8878e7b0ddf2301855a13bef15d2a35d431ac709ac2ce02225fefbbe"}
Sep 18 20:05:50.021000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.020891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm" event={"ID":"659002d6-e06b-4bce-9329-61b69116bb31","Type":"ContainerStarted","Data":"fab51cecc14f75f60b7c9aead56d7ee00cce5052774d6a96a0760615a786521c"}
Sep 18 20:05:50.021000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.020953    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm" event={"ID":"659002d6-e06b-4bce-9329-61b69116bb31","Type":"ContainerStarted","Data":"69df123d6d3e4de328667eaf57d79e505b59d5667ee75cbcd930dcde4ef8a2b7"}
Sep 18 20:05:50.021000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.020981    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm" event={"ID":"659002d6-e06b-4bce-9329-61b69116bb31","Type":"ContainerStarted","Data":"9c7255f4451d0ffab0831acf44153039d5e28b931f9fc5a3d9fdaa2186be937c"}
Sep 18 20:05:50.060876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:50.060822    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-84c885658c-mj9nm" podStartSLOduration=1.640928194 podStartE2EDuration="3.06079977s" totalImagesPullingTime="1.419871576s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:50.059951159 +0000 UTC m=+294.189253123" watchObservedRunningTime="2026-09-18 20:05:50.06079977 +0000 UTC m=+294.190101722"
Sep 18 20:05:51.030729 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:51.030677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5fhs" event={"ID":"c2993b00-deeb-400f-9c59-306ba1adc201","Type":"ContainerStarted","Data":"f7c79bdb5e5b83c10f9b0dff28484c3ca31d5d71d486a1c7561b68e1e5428574"}
Sep 18 20:05:51.031157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:51.030797    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-m5fhs" event={"ID":"c2993b00-deeb-400f-9c59-306ba1adc201","Type":"ContainerStarted","Data":"e5ae7ed35b77ac76232b82ccd4dc18e2b82f4c48b6f2d5c0e5f9b59085d3591a"}
Sep 18 20:05:51.053466 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:51.053411    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-m5fhs" podStartSLOduration=2.614488091 podStartE2EDuration="4.053394878s" totalImagesPullingTime="1.356592482s" totalInitContainerRuntime="82.314305ms" isStatefulPod=true podCreationTimestamp="2026-09-18 20:05:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:51.051902432 +0000 UTC m=+295.181204383" watchObservedRunningTime="2026-09-18 20:05:51.053394878 +0000 UTC m=+295.182696828"
Sep 18 20:05:51.788755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:51.788715    2566 patch_prober.go:28] interesting pod/console-7585545484-nddg2 container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused" start-of-body=
Sep 18 20:05:51.788943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:51.788769    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7585545484-nddg2" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console" probeResult="failure" output="Get \"https://10.132.0.17:8443/health\": dial tcp 10.132.0.17:8443: connect: connection refused"
Sep 18 20:05:52.459129 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.459085    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:05:52.486199 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.486163    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:05:52.489554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.489533    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.495917 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.495894    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 20:05:52.498933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.498905    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:05:52.540342 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540301    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540342 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540338    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540386    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-psq6m\" (UniqueName: \"kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540419    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540441    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540467    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.540542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.540484    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641760    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641780    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641818    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-psq6m\" (UniqueName: \"kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.641927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.641867    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.642259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.642031    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.642698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.642652    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.642698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.642681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.642829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.642762    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.642829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.642777    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.644452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.644421    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.644540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.644485    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.649364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.649338    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-psq6m\" (UniqueName: \"kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m\") pod \"console-b7984474f-2sgrk\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") " pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:52.800251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:52.800148    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:05:53.079525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.079430    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:05:53.082528 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:53.082487    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode466a913_d3ed_4434_9a63_f870b07e5771.slice/crio-46ffa9e4674dc33d62205c3db7f0d45e5e2c26097d226f32343737e510447e84 WatchSource:0}: Error finding container 46ffa9e4674dc33d62205c3db7f0d45e5e2c26097d226f32343737e510447e84: Status 404 returned error can't find the container with id 46ffa9e4674dc33d62205c3db7f0d45e5e2c26097d226f32343737e510447e84
Sep 18 20:05:53.348472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.348382    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:05:53.352459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.352430    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.355005 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.354982    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\""
Sep 18 20:05:53.355303 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.355136    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-pcxtf\""
Sep 18 20:05:53.355387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.355360    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\""
Sep 18 20:05:53.355387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.355379    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\""
Sep 18 20:05:53.356120 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356096    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\""
Sep 18 20:05:53.356245 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356206    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\""
Sep 18 20:05:53.356339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356325    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\""
Sep 18 20:05:53.356489 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356406    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\""
Sep 18 20:05:53.356489 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356458    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\""
Sep 18 20:05:53.356489 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356483    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
Sep 18 20:05:53.356678 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356517    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\""
Sep 18 20:05:53.356678 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356568    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\""
Sep 18 20:05:53.356776 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356758    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-8eepr0updud2h\""
Sep 18 20:05:53.356942 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356919    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\""
Sep 18 20:05:53.357048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.356944    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\""
Sep 18 20:05:53.360384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.360003    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\""
Sep 18 20:05:53.368273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.368244    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\""
Sep 18 20:05:53.372879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.372831    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:05:53.449735 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.449662    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.449735 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.449742    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.449793    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.449820    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.449940    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450007    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450042    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450188    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450210    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450238    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450261    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450420    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450480    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450519    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450544    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9txrb\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-kube-api-access-9txrb\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450648    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450690    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450729    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.450848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.450754    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.551695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.551695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551688    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551762    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551783    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551816    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551839    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551864    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551888    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551910    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.551977    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552003    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552023    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9txrb\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-kube-api-access-9txrb\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552091    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552116    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552144    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552169    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.552269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552196    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.552225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.553003    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.553001    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.553307    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.553528    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.553704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.553677    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.554243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.554221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.555318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.555290    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.555872 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.555844    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557128    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-web-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557311    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557366    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557440    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.557702 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.557650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/f4c43adc-b596-4d86-8fee-bac0c6cd854c-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.558148 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.558126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.559243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.559202    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.560073 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.559650    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/f4c43adc-b596-4d86-8fee-bac0c6cd854c-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.560073 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.559845    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f4c43adc-b596-4d86-8fee-bac0c6cd854c-config-out\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.563187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.563166    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9txrb\" (UniqueName: \"kubernetes.io/projected/f4c43adc-b596-4d86-8fee-bac0c6cd854c-kube-api-access-9txrb\") pod \"prometheus-k8s-0\" (UID: \"f4c43adc-b596-4d86-8fee-bac0c6cd854c\") " pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.670938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.670515    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:05:53.962847 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.962820    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"]
Sep 18 20:05:53.964507 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:05:53.964479    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4c43adc_b596_4d86_8fee_bac0c6cd854c.slice/crio-2782b2acd5e87591ce9f04a0ff416b7e3564ebb742baad8493ae10441eee61cc WatchSource:0}: Error finding container 2782b2acd5e87591ce9f04a0ff416b7e3564ebb742baad8493ae10441eee61cc: Status 404 returned error can't find the container with id 2782b2acd5e87591ce9f04a0ff416b7e3564ebb742baad8493ae10441eee61cc
Sep 18 20:05:53.982396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:53.982362    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qvdjd"
Sep 18 20:05:54.046903 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:54.046863    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7984474f-2sgrk" event={"ID":"e466a913-d3ed-4434-9a63-f870b07e5771","Type":"ContainerStarted","Data":"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"}
Sep 18 20:05:54.047102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:54.046959    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7984474f-2sgrk" event={"ID":"e466a913-d3ed-4434-9a63-f870b07e5771","Type":"ContainerStarted","Data":"46ffa9e4674dc33d62205c3db7f0d45e5e2c26097d226f32343737e510447e84"}
Sep 18 20:05:54.048293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:54.048269    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"2782b2acd5e87591ce9f04a0ff416b7e3564ebb742baad8493ae10441eee61cc"}
Sep 18 20:05:54.063165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:54.063104    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-b7984474f-2sgrk" podStartSLOduration=2.063086623 podStartE2EDuration="2.063086623s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:05:54.062209101 +0000 UTC m=+298.191511053" watchObservedRunningTime="2026-09-18 20:05:54.063086623 +0000 UTC m=+298.192388574"
Sep 18 20:05:54.074836 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:05:54.074787    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-4jr7x" podUID="d7a6ca69-b215-437a-9191-d46540f76181"
Sep 18 20:05:56.057996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:56.057961    2566 generic.go:356] "Generic (PLEG): container finished" podID="f4c43adc-b596-4d86-8fee-bac0c6cd854c" containerID="005ceeba3aefdc797bfcc6ca4f0269ccb9f5b4add7adbdd643080c22341dafec" exitCode=0
Sep 18 20:05:56.058523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:56.058017    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerDied","Data":"005ceeba3aefdc797bfcc6ca4f0269ccb9f5b4add7adbdd643080c22341dafec"}
Sep 18 20:05:58.004284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:58.004243    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:05:58.004739 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:58.004360    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:05:58.013573 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:58.013535    2566 kubelet.go:1648] "Image garbage collection succeeded"
Sep 18 20:05:59.458050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:05:59.458016    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:06:00.080488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:00.080440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"b68b1cb49ab0a8a8e855242efd6f01cd67430302dc9669221abc60791839d670"}
Sep 18 20:06:00.080687 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:00.080507    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"24092e15e92ad643be84e4b154f5e5c58fe1e350eb19c425efd1ee1a01ca3fcf"}
Sep 18 20:06:01.268104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:01.267487    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:06:01.277205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:01.277172    2566 patch_prober.go:28] interesting pod/image-registry-65c945669c-gwd5f container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
Sep 18 20:06:01.277381 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:01.277228    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:06:02.091453 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.091418    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"6bdd1c9e0f35ed8f59e7d44178200ebdd0a8a448967cd258f581e859a9b2c0eb"}
Sep 18 20:06:02.091609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.091484    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"e516923f01d4bc46259782ecf0283b9a76a2a50e661842b5d42d9234c51d1ec5"}
Sep 18 20:06:02.091609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.091512    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"77c7ed8b7cbb6af5d7ecd1fa26d0bbd298f851b3b4eb3f0cd44c515c5f4fba97"}
Sep 18 20:06:02.091609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.091536    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"f4c43adc-b596-4d86-8fee-bac0c6cd854c","Type":"ContainerStarted","Data":"74ba13936f7a62c3d2165e9bdd826af0203b2c0e5b75e8502ad54219c1b6c9d7"}
Sep 18 20:06:02.118014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.117956    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8667473169999997 podStartE2EDuration="9.117940598s" totalImagesPullingTime="6.231582498s" totalInitContainerRuntime="19.610783ms" isStatefulPod=false podCreationTimestamp="2026-09-18 20:05:53 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:06:02.116350255 +0000 UTC m=+306.245652230" watchObservedRunningTime="2026-09-18 20:06:02.117940598 +0000 UTC m=+306.247242548"
Sep 18 20:06:02.800780 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.800747    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:02.801146 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.800830    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:02.805347 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:02.805327    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:03.099052 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:03.098992    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:03.671102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:03.671065    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:06:05.074120 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:05.074087    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:06:10.261130 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:10.261093    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:06:11.272181 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:11.272150    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:06:17.478779 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.478735    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7585545484-nddg2" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console" containerID="cri-o://7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824" gracePeriod=15
Sep 18 20:06:17.719698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.719673    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:06:17.866405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.866315    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.866630 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.866611    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.866757 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.866744    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44r6j\" (UniqueName: \"kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.866863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.866851    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.866991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.866979    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.867140 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.867126    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca\") pod \"129315b1-277d-48a3-8648-fcb2d1be3395\" (UID: \"129315b1-277d-48a3-8648-fcb2d1be3395\") "
Sep 18 20:06:17.868027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.867528    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:17.868027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.867784    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca" (OuterVolumeSpecName: "service-ca") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:17.868641 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.868615    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config" (OuterVolumeSpecName: "console-config") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:17.872163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.872043    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:17.872163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.872096    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j" (OuterVolumeSpecName: "kube-api-access-44r6j") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "kube-api-access-44r6j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:06:17.872163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.872140    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "129315b1-277d-48a3-8648-fcb2d1be3395" (UID: "129315b1-277d-48a3-8648-fcb2d1be3395"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:17.968252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968218    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44r6j\" (UniqueName: \"kubernetes.io/projected/129315b1-277d-48a3-8648-fcb2d1be3395-kube-api-access-44r6j\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:17.968252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968246    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-serving-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:17.968252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968258    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/129315b1-277d-48a3-8648-fcb2d1be3395-console-oauth-config\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:17.968457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968270    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:17.968457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968282    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-oauth-serving-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:17.968457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:17.968293    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/129315b1-277d-48a3-8648-fcb2d1be3395-console-config\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:18.149401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.149324    2566 generic.go:356] "Generic (PLEG): container finished" podID="129315b1-277d-48a3-8648-fcb2d1be3395" containerID="7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824" exitCode=0
Sep 18 20:06:18.149401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.149376    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7585545484-nddg2"
Sep 18 20:06:18.149738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.149365    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7585545484-nddg2" event={"ID":"129315b1-277d-48a3-8648-fcb2d1be3395","Type":"ContainerDied","Data":"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"}
Sep 18 20:06:18.149738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.149480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7585545484-nddg2" event={"ID":"129315b1-277d-48a3-8648-fcb2d1be3395","Type":"ContainerDied","Data":"c922dc66e1b72d341be1102f6a4be2d25606b016bd0b009cf7c53f27926d26b8"}
Sep 18 20:06:18.149738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.149502    2566 scope.go:117] "RemoveContainer" containerID="7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"
Sep 18 20:06:18.157668 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.157649    2566 scope.go:117] "RemoveContainer" containerID="7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"
Sep 18 20:06:18.157912 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:06:18.157890    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824\": container with ID starting with 7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824 not found: ID does not exist" containerID="7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"
Sep 18 20:06:18.157956 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.157919    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824"} err="failed to get container status \"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824\": rpc error: code = NotFound desc = could not find container \"7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824\": container with ID starting with 7cb7f676b0d1cc0b193b6ed6f5e38d6e446cf937d08a7b455da8e25a9296f824 not found: ID does not exist"
Sep 18 20:06:18.164559 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.164540    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:06:18.166570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:18.166554    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7585545484-nddg2"]
Sep 18 20:06:20.079038 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:20.079000    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" path="/var/lib/kubelet/pods/129315b1-277d-48a3-8648-fcb2d1be3395/volumes"
Sep 18 20:06:22.171862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:22.171830    2566 generic.go:356] "Generic (PLEG): container finished" podID="5b564ccd-d8f3-4d3a-91f6-bebb2043fccb" containerID="feb1be49bad7bf440200a19811bad1e197cfce76551a03e63e307096d4559b66" exitCode=0
Sep 18 20:06:22.172231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:22.171909    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc" event={"ID":"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb","Type":"ContainerDied","Data":"feb1be49bad7bf440200a19811bad1e197cfce76551a03e63e307096d4559b66"}
Sep 18 20:06:22.172287 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:22.172268    2566 scope.go:117] "RemoveContainer" containerID="feb1be49bad7bf440200a19811bad1e197cfce76551a03e63e307096d4559b66"
Sep 18 20:06:23.178509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:23.178476    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-6f9c644dd9-2pxgc" event={"ID":"5b564ccd-d8f3-4d3a-91f6-bebb2043fccb","Type":"ContainerStarted","Data":"204f48f755f63009ddd7bb499a7a6049c28a4c8d0c675fea9eb37ffe70399c42"}
Sep 18 20:06:26.292865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.292828    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerName="registry" containerID="cri-o://e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167" gracePeriod=30
Sep 18 20:06:26.529149 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.529127    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:06:26.636874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636818    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.636874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636848    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6k7jw\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.636874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636872    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636896    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636928    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636949    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636965    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.636991    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca\") pod \"a5ab2815-9837-437a-bae0-c11f02acaac1\" (UID: \"a5ab2815-9837-437a-bae0-c11f02acaac1\") "
Sep 18 20:06:26.637334 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.637259    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:26.637614 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.637589    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:26.639361 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.639326    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:06:26.639468 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.639413    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:06:26.639610 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.639583    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:26.639677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.639644    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:26.639736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.639672    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw" (OuterVolumeSpecName: "kube-api-access-6k7jw") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "kube-api-access-6k7jw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:06:26.645531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.645507    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a5ab2815-9837-437a-bae0-c11f02acaac1" (UID: "a5ab2815-9837-437a-bae0-c11f02acaac1"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:06:26.738112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738090    2566 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a5ab2815-9837-437a-bae0-c11f02acaac1-ca-trust-extracted\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738111    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6k7jw\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-kube-api-access-6k7jw\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738121    2566 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-certificates\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738130    2566 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-bound-sa-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738138    2566 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a5ab2815-9837-437a-bae0-c11f02acaac1-registry-tls\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738146    2566 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-installation-pull-secrets\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738154    2566 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a5ab2815-9837-437a-bae0-c11f02acaac1-image-registry-private-configuration\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:26.738225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:26.738162    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a5ab2815-9837-437a-bae0-c11f02acaac1-trusted-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:27.196335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.196311    2566 generic.go:356] "Generic (PLEG): container finished" podID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerID="e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167" exitCode=0
Sep 18 20:06:27.196471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.196360    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-65c945669c-gwd5f"
Sep 18 20:06:27.196471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.196398    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" event={"ID":"a5ab2815-9837-437a-bae0-c11f02acaac1","Type":"ContainerDied","Data":"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"}
Sep 18 20:06:27.196471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.196466    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-65c945669c-gwd5f" event={"ID":"a5ab2815-9837-437a-bae0-c11f02acaac1","Type":"ContainerDied","Data":"9a7262e16393973d9ddac7b8dbf3b377a6dd5a64bce943e0f91a562851575224"}
Sep 18 20:06:27.196571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.196488    2566 scope.go:117] "RemoveContainer" containerID="e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"
Sep 18 20:06:27.204709 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.204696    2566 scope.go:117] "RemoveContainer" containerID="e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"
Sep 18 20:06:27.204962 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:06:27.204945    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167\": container with ID starting with e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167 not found: ID does not exist" containerID="e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"
Sep 18 20:06:27.205014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.204969    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167"} err="failed to get container status \"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167\": rpc error: code = NotFound desc = could not find container \"e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167\": container with ID starting with e39d1618fcb6307d75a88bf3df4d478ae1f7ff87c1eebb5840be58dbeef44167 not found: ID does not exist"
Sep 18 20:06:27.215881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.215862    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:06:27.219573 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:27.219553    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-65c945669c-gwd5f"]
Sep 18 20:06:28.079032 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:28.079002    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" path="/var/lib/kubelet/pods/a5ab2815-9837-437a-bae0-c11f02acaac1/volumes"
Sep 18 20:06:35.279552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.279513    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-b7984474f-2sgrk" podUID="e466a913-d3ed-4434-9a63-f870b07e5771" containerName="console" containerID="cri-o://907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169" gracePeriod=15
Sep 18 20:06:35.546883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.546487    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:35.608386 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608356    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608406    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608446    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608466    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608488    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-psq6m\" (UniqueName: \"kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608539    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608818 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608580    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca\") pod \"e466a913-d3ed-4434-9a63-f870b07e5771\" (UID: \"e466a913-d3ed-4434-9a63-f870b07e5771\") "
Sep 18 20:06:35.608873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.608848    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:35.609122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.609026    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config" (OuterVolumeSpecName: "console-config") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:35.609122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.609031    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:35.609288 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.609138    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca" (OuterVolumeSpecName: "service-ca") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:06:35.610666 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.610635    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:35.610777 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.610700    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m" (OuterVolumeSpecName: "kube-api-access-psq6m") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "kube-api-access-psq6m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:06:35.610777 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.610738    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "e466a913-d3ed-4434-9a63-f870b07e5771" (UID: "e466a913-d3ed-4434-9a63-f870b07e5771"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:06:35.709443 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709411    2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-console-config\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709443 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709442    2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709453    2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-oauth-config\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709467    2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/e466a913-d3ed-4434-9a63-f870b07e5771-console-serving-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709479    2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-oauth-serving-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709490    2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e466a913-d3ed-4434-9a63-f870b07e5771-trusted-ca-bundle\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:35.709637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:35.709500    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-psq6m\" (UniqueName: \"kubernetes.io/projected/e466a913-d3ed-4434-9a63-f870b07e5771-kube-api-access-psq6m\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:06:36.232124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.232089    2566 generic.go:356] "Generic (PLEG): container finished" podID="e466a913-d3ed-4434-9a63-f870b07e5771" containerID="907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169" exitCode=0
Sep 18 20:06:36.232275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.232147    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-b7984474f-2sgrk"
Sep 18 20:06:36.232275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.232172    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7984474f-2sgrk" event={"ID":"e466a913-d3ed-4434-9a63-f870b07e5771","Type":"ContainerDied","Data":"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"}
Sep 18 20:06:36.232275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.232227    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-b7984474f-2sgrk" event={"ID":"e466a913-d3ed-4434-9a63-f870b07e5771","Type":"ContainerDied","Data":"46ffa9e4674dc33d62205c3db7f0d45e5e2c26097d226f32343737e510447e84"}
Sep 18 20:06:36.232275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.232250    2566 scope.go:117] "RemoveContainer" containerID="907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"
Sep 18 20:06:36.240792 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.240775    2566 scope.go:117] "RemoveContainer" containerID="907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"
Sep 18 20:06:36.241137 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:06:36.241111    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169\": container with ID starting with 907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169 not found: ID does not exist" containerID="907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"
Sep 18 20:06:36.241218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.241142    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169"} err="failed to get container status \"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169\": rpc error: code = NotFound desc = could not find container \"907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169\": container with ID starting with 907614829daed8530fd1a3db86b8621f21ec270d357b653f01fe5491dcfb5169 not found: ID does not exist"
Sep 18 20:06:36.248587 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.248565    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:06:36.251116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:36.251096    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-b7984474f-2sgrk"]
Sep 18 20:06:37.238856 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:37.238826    2566 generic.go:356] "Generic (PLEG): container finished" podID="e39a38af-479b-4c64-91d2-70f12b9525b4" containerID="01b005b822b628d48bf6b1d5485dd4e5c9d6ae7deccf6714b7d1c5ede65605b8" exitCode=0
Sep 18 20:06:37.239241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:37.238905    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk" event={"ID":"e39a38af-479b-4c64-91d2-70f12b9525b4","Type":"ContainerDied","Data":"01b005b822b628d48bf6b1d5485dd4e5c9d6ae7deccf6714b7d1c5ede65605b8"}
Sep 18 20:06:37.239241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:37.239181    2566 scope.go:117] "RemoveContainer" containerID="01b005b822b628d48bf6b1d5485dd4e5c9d6ae7deccf6714b7d1c5ede65605b8"
Sep 18 20:06:38.080937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:38.080896    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e466a913-d3ed-4434-9a63-f870b07e5771" path="/var/lib/kubelet/pods/e466a913-d3ed-4434-9a63-f870b07e5771/volumes"
Sep 18 20:06:38.244116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:38.244084    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-55889f7b47-k6krk" event={"ID":"e39a38af-479b-4c64-91d2-70f12b9525b4","Type":"ContainerStarted","Data":"d243aa351b213e1b6ea42918791976de8eb65fb2b1885d1c7f2ce6f8a3c7e313"}
Sep 18 20:06:53.671865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:53.671830    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:06:53.688210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:53.688186    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:06:54.325555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:06:54.325518    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0"
Sep 18 20:07:11.889179 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:11.889149    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:07:11.891311 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:11.891290    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d7a6ca69-b215-437a-9191-d46540f76181-metrics-certs\") pod \"network-metrics-daemon-4jr7x\" (UID: \"d7a6ca69-b215-437a-9191-d46540f76181\") " pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:07:11.976918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:11.976896    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-ms6pd\""
Sep 18 20:07:11.985315 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:11.985298    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-4jr7x"
Sep 18 20:07:12.254078 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:12.254018    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-4jr7x"]
Sep 18 20:07:12.257034 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:07:12.257001    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd7a6ca69_b215_437a_9191_d46540f76181.slice/crio-008e6496965f092f130161c7ed451743b4847eccdf506f7fb77a4db1b2164ee7 WatchSource:0}: Error finding container 008e6496965f092f130161c7ed451743b4847eccdf506f7fb77a4db1b2164ee7: Status 404 returned error can't find the container with id 008e6496965f092f130161c7ed451743b4847eccdf506f7fb77a4db1b2164ee7
Sep 18 20:07:12.378250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:12.378219    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jr7x" event={"ID":"d7a6ca69-b215-437a-9191-d46540f76181","Type":"ContainerStarted","Data":"008e6496965f092f130161c7ed451743b4847eccdf506f7fb77a4db1b2164ee7"}
Sep 18 20:07:13.384868 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:13.384839    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jr7x" event={"ID":"d7a6ca69-b215-437a-9191-d46540f76181","Type":"ContainerStarted","Data":"cca9e3d938e625edde1b49b804a9b129f14f99566ae1ee0e53b02fdfb3c037d7"}
Sep 18 20:07:14.390640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:14.390607    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-4jr7x" event={"ID":"d7a6ca69-b215-437a-9191-d46540f76181","Type":"ContainerStarted","Data":"22852664aa5b5090e9fb66d1c8fd8e515cf257ebc6dd09cf914ce79d9b2cc430"}
Sep 18 20:07:14.406104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:07:14.406023    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-4jr7x" podStartSLOduration=375.389615394 podStartE2EDuration="6m16.406007445s" totalImagesPullingTime="1.016392051s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:00:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:07:14.404836599 +0000 UTC m=+378.534138549" watchObservedRunningTime="2026-09-18 20:07:14.406007445 +0000 UTC m=+378.535309395"
Sep 18 20:08:30.767894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.767863    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8bgd"]
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768315    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerName="registry"
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768333    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerName="registry"
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768352    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console"
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768363    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console"
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768373    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e466a913-d3ed-4434-9a63-f870b07e5771" containerName="console"
Sep 18 20:08:30.768436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768382    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e466a913-d3ed-4434-9a63-f870b07e5771" containerName="console"
Sep 18 20:08:30.768725 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768470    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e466a913-d3ed-4434-9a63-f870b07e5771" containerName="console"
Sep 18 20:08:30.768725 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768487    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="129315b1-277d-48a3-8648-fcb2d1be3395" containerName="console"
Sep 18 20:08:30.768725 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.768497    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5ab2815-9837-437a-bae0-c11f02acaac1" containerName="registry"
Sep 18 20:08:30.771564 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.771544    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.773949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.773924    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 18 20:08:30.774095 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.774073    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 18 20:08:30.774829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.774815    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-7xdjq\""
Sep 18 20:08:30.778251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.778227    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8bgd"]
Sep 18 20:08:30.879995 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.879968    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bt9nb\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-kube-api-access-bt9nb\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.880187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.880028    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.980606 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.980567    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.980774 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.980633    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bt9nb\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-kube-api-access-bt9nb\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.988309 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.988284    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-bound-sa-token\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:30.988443 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:30.988426    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bt9nb\" (UniqueName: \"kubernetes.io/projected/25b9bf8e-9def-4dbc-aa19-7c23fcd467b7-kube-api-access-bt9nb\") pod \"cert-manager-55b4d954d5-c8bgd\" (UID: \"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7\") " pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:31.092604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:31.092541    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-c8bgd"
Sep 18 20:08:31.361205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:31.361118    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-c8bgd"]
Sep 18 20:08:31.365986 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:08:31.365955    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod25b9bf8e_9def_4dbc_aa19_7c23fcd467b7.slice/crio-8913f40c44257a7774205ac73eb759df6e3165f6b6deb9b387a5605f8055ed28 WatchSource:0}: Error finding container 8913f40c44257a7774205ac73eb759df6e3165f6b6deb9b387a5605f8055ed28: Status 404 returned error can't find the container with id 8913f40c44257a7774205ac73eb759df6e3165f6b6deb9b387a5605f8055ed28
Sep 18 20:08:31.695368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:31.695250    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c8bgd" event={"ID":"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7","Type":"ContainerStarted","Data":"8913f40c44257a7774205ac73eb759df6e3165f6b6deb9b387a5605f8055ed28"}
Sep 18 20:08:35.713386 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:35.713343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-c8bgd" event={"ID":"25b9bf8e-9def-4dbc-aa19-7c23fcd467b7","Type":"ContainerStarted","Data":"d8774d57ca3ccdcef30199611d3bdf19f8244cfaf912bd0e80e9a5e1ba6afe4b"}
Sep 18 20:08:35.731014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:35.730956    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-c8bgd" podStartSLOduration=2.4327387 podStartE2EDuration="5.73094056s" totalImagesPullingTime="3.29820186s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:08:35.730285283 +0000 UTC m=+459.859587230" watchObservedRunningTime="2026-09-18 20:08:35.73094056 +0000 UTC m=+459.860242511"
Sep 18 20:08:55.898019 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.897980    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"]
Sep 18 20:08:55.903439 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.903410    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:55.908144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.908120    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 18 20:08:55.908292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.908207    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 18 20:08:55.908292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.908168    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-ws545\""
Sep 18 20:08:55.908292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.908156    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 18 20:08:55.908633 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.908612    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 18 20:08:55.919938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:55.919915    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"]
Sep 18 20:08:56.091538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.091493    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.091715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.091543    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.091715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.091682    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jprx\" (UniqueName: \"kubernetes.io/projected/68d37f52-8f5c-4c28-9f08-ba731575d78d-kube-api-access-5jprx\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.091819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.091727    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.091819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.091757    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192479    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jprx\" (UniqueName: \"kubernetes.io/projected/68d37f52-8f5c-4c28-9f08-ba731575d78d-kube-api-access-5jprx\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192532    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192592    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.192977 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.192953    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-manifests\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.195039 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.195000    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/68d37f52-8f5c-4c28-9f08-ba731575d78d-tmp\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.195238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.195040    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-webhook-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.195285 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.195234    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/68d37f52-8f5c-4c28-9f08-ba731575d78d-apiservice-cert\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.202640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.202615    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jprx\" (UniqueName: \"kubernetes.io/projected/68d37f52-8f5c-4c28-9f08-ba731575d78d-kube-api-access-5jprx\") pod \"opendatahub-operator-controller-manager-668c7cd96d-98279\" (UID: \"68d37f52-8f5c-4c28-9f08-ba731575d78d\") " pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.216936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.216911    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:08:56.490999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.490974    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"]
Sep 18 20:08:56.492971 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:08:56.492939    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68d37f52_8f5c_4c28_9f08_ba731575d78d.slice/crio-c688e48c570ced72fbd8191173f239b99ec3329c8e2d931b3012157c6ed0069e WatchSource:0}: Error finding container c688e48c570ced72fbd8191173f239b99ec3329c8e2d931b3012157c6ed0069e: Status 404 returned error can't find the container with id c688e48c570ced72fbd8191173f239b99ec3329c8e2d931b3012157c6ed0069e
Sep 18 20:08:56.797473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:08:56.797391    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279" event={"ID":"68d37f52-8f5c-4c28-9f08-ba731575d78d","Type":"ContainerStarted","Data":"c688e48c570ced72fbd8191173f239b99ec3329c8e2d931b3012157c6ed0069e"}
Sep 18 20:09:00.819300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:00.819260    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279" event={"ID":"68d37f52-8f5c-4c28-9f08-ba731575d78d","Type":"ContainerStarted","Data":"8b162bb1daa1b1f54df71218554e3eee8297446a83eb0c85e569b9f8ec90f7f3"}
Sep 18 20:09:01.825026 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:01.824990    2566 generic.go:356] "Generic (PLEG): container finished" podID="68d37f52-8f5c-4c28-9f08-ba731575d78d" containerID="8b162bb1daa1b1f54df71218554e3eee8297446a83eb0c85e569b9f8ec90f7f3" exitCode=0
Sep 18 20:09:01.825552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:01.825092    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279" event={"ID":"68d37f52-8f5c-4c28-9f08-ba731575d78d","Type":"ContainerDied","Data":"8b162bb1daa1b1f54df71218554e3eee8297446a83eb0c85e569b9f8ec90f7f3"}
Sep 18 20:09:02.831517 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:02.831483    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279" event={"ID":"68d37f52-8f5c-4c28-9f08-ba731575d78d","Type":"ContainerStarted","Data":"a0d6719782eaee3c340141b52d7c87a8c642283272b8c3e78ba83ffb8048fbf4"}
Sep 18 20:09:02.831935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:02.831602    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:09:02.853550 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:02.853507    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279" podStartSLOduration=3.045983658 podStartE2EDuration="7.853493362s" totalImagesPullingTime="3.770260821s" totalInitContainerRuntime="1.037248883s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:08:55 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:02.852128606 +0000 UTC m=+486.981430551" watchObservedRunningTime="2026-09-18 20:09:02.853493362 +0000 UTC m=+486.982795313"
Sep 18 20:09:13.838374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:13.838342    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-668c7cd96d-98279"
Sep 18 20:09:22.033967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.033926    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"]
Sep 18 20:09:22.037459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.037436    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.039986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.039963    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 18 20:09:22.040191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.039967    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-c4mrc\""
Sep 18 20:09:22.051265 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.051240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"]
Sep 18 20:09:22.084270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084389 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084275    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084389 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084294    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084389 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084317    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084382    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084446    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084462    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.084514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.084478    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qg55w\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-kube-api-access-qg55w\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.185451 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185427    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.185545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.185545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185489    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.185545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185505    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.185545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185521    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qg55w\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-kube-api-access-qg55w\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185823    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185857    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185920    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185953    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.185975    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.186176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.186342    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-data\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.186718 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.186692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.187854 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.187831    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.188216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.188195    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.193197 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.193175    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qg55w\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-kube-api-access-qg55w\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.193272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.193252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2-istio-token\") pod \"data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x\" (UID: \"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.353350 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.353295    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:22.629941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.629907    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"]
Sep 18 20:09:22.632842 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:09:22.632805    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f7a7cbc_36de_48bb_9b53_7f6f0c1e8ec2.slice/crio-2c90510bd3044881b57c1e573aa129d533622780cf0147872999f30436646702 WatchSource:0}: Error finding container 2c90510bd3044881b57c1e573aa129d533622780cf0147872999f30436646702: Status 404 returned error can't find the container with id 2c90510bd3044881b57c1e573aa129d533622780cf0147872999f30436646702
Sep 18 20:09:22.920667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:22.920597    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x" event={"ID":"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2","Type":"ContainerStarted","Data":"2c90510bd3044881b57c1e573aa129d533622780cf0147872999f30436646702"}
Sep 18 20:09:25.560441 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:25.560403    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:09:25.560826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:25.560495    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:09:25.560826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:25.560545    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:09:25.936109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:25.936031    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x" event={"ID":"9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2","Type":"ContainerStarted","Data":"5b2eb89dcec8295a267be38e1c1e24777f2e8b5f3151157e0065a86b922fb43f"}
Sep 18 20:09:25.955908 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:25.955861    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x" podStartSLOduration=1.031087289 podStartE2EDuration="3.955844334s" totalImagesPullingTime="2.924757045s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:25.954623476 +0000 UTC m=+510.083925417" watchObservedRunningTime="2026-09-18 20:09:25.955844334 +0000 UTC m=+510.085146285"
Sep 18 20:09:26.353677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:26.353650    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:26.357886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:26.357867    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:26.940581 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:26.940536    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:26.941433 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:26.941413    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x"
Sep 18 20:09:34.771287 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.771253    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"]
Sep 18 20:09:34.779306 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.779284    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.783228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783201    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
Sep 18 20:09:34.783332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783202    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
Sep 18 20:09:34.783332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783202    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
Sep 18 20:09:34.783332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783202    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-bbght\""
Sep 18 20:09:34.783332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783211    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
Sep 18 20:09:34.783551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.783266    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
Sep 18 20:09:34.788444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.788424    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"]
Sep 18 20:09:34.882792 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.882765    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rp2m\" (UniqueName: \"kubernetes.io/projected/0ae879f3-7285-44eb-8bac-6c2549860955-kube-api-access-9rp2m\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.882909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.882808    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.882909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.882840    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae879f3-7285-44eb-8bac-6c2549860955-manager-config\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.882909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.882890    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.984074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.984029    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rp2m\" (UniqueName: \"kubernetes.io/projected/0ae879f3-7285-44eb-8bac-6c2549860955-kube-api-access-9rp2m\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.984252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.984082    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.984252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.984201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae879f3-7285-44eb-8bac-6c2549860955-manager-config\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.984371 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.984270    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.985110 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.985082    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/0ae879f3-7285-44eb-8bac-6c2549860955-manager-config\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.986706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.986686    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-metrics-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.986959 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.986938    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/0ae879f3-7285-44eb-8bac-6c2549860955-cert\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:34.992270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:34.992250    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rp2m\" (UniqueName: \"kubernetes.io/projected/0ae879f3-7285-44eb-8bac-6c2549860955-kube-api-access-9rp2m\") pod \"lws-controller-manager-76d6fbff56-zrxqm\" (UID: \"0ae879f3-7285-44eb-8bac-6c2549860955\") " pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:35.089603 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:35.089524    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:35.363635 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:35.363611    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"]
Sep 18 20:09:35.366298 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:09:35.366266    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ae879f3_7285_44eb_8bac_6c2549860955.slice/crio-50d2ba0b1c8b06c2f5152dea4560418e6973b0d435d79542a7c16c10acc5108a WatchSource:0}: Error finding container 50d2ba0b1c8b06c2f5152dea4560418e6973b0d435d79542a7c16c10acc5108a: Status 404 returned error can't find the container with id 50d2ba0b1c8b06c2f5152dea4560418e6973b0d435d79542a7c16c10acc5108a
Sep 18 20:09:35.982746 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:35.982704    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm" event={"ID":"0ae879f3-7285-44eb-8bac-6c2549860955","Type":"ContainerStarted","Data":"50d2ba0b1c8b06c2f5152dea4560418e6973b0d435d79542a7c16c10acc5108a"}
Sep 18 20:09:36.220357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.220311    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"]
Sep 18 20:09:36.223243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.223219    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.225872 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.225847    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-mgv6t\""
Sep 18 20:09:36.225985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.225874    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\""
Sep 18 20:09:36.237675 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.237614    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"]
Sep 18 20:09:36.239983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.239963    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"]
Sep 18 20:09:36.243665 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.243570    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.247727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.247518    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\""
Sep 18 20:09:36.249698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.249400    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-5dvjl\""
Sep 18 20:09:36.249698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.249597    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"]
Sep 18 20:09:36.294853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.294819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.295008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.294875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7z4g\" (UniqueName: \"kubernetes.io/projected/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-kube-api-access-c7z4g\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.295008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.294916    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h65tx\" (UniqueName: \"kubernetes.io/projected/d260a66a-c621-4346-aaf0-e3b0f5374480-kube-api-access-h65tx\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.295008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.294942    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.396020 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.395984    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.396176 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.396034    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7z4g\" (UniqueName: \"kubernetes.io/projected/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-kube-api-access-c7z4g\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.396176 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.396079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h65tx\" (UniqueName: \"kubernetes.io/projected/d260a66a-c621-4346-aaf0-e3b0f5374480-kube-api-access-h65tx\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.396176 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.396121    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.396326 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:09:36.396213    2566 secret.go:189] Couldn't get secret opendatahub/llmisvc-webhook-server-cert: secret "llmisvc-webhook-server-cert" not found
Sep 18 20:09:36.396326 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:09:36.396266    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert podName:d260a66a-c621-4346-aaf0-e3b0f5374480 nodeName:}" failed. No retries permitted until 2026-09-18 20:09:36.8962471 +0000 UTC m=+521.025549032 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert") pod "llmisvc-controller-manager-777c6bd7d5-728vz" (UID: "d260a66a-c621-4346-aaf0-e3b0f5374480") : secret "llmisvc-webhook-server-cert" not found
Sep 18 20:09:36.400003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.399982    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-cert\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.408809 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.408785    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h65tx\" (UniqueName: \"kubernetes.io/projected/d260a66a-c621-4346-aaf0-e3b0f5374480-kube-api-access-h65tx\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.422795 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.422767    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7z4g\" (UniqueName: \"kubernetes.io/projected/2be4f5eb-ab90-4b6d-b7a9-4e147f52be11-kube-api-access-c7z4g\") pod \"kserve-controller-manager-d6f4b65b8-2ctb4\" (UID: \"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.538988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.538895    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:36.821556 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.821484    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"]
Sep 18 20:09:36.900212 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.900183    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:36.902615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:36.902593    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d260a66a-c621-4346-aaf0-e3b0f5374480-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-728vz\" (UID: \"d260a66a-c621-4346-aaf0-e3b0f5374480\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:37.157960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:37.157881    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:37.484682 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:09:37.484650    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2be4f5eb_ab90_4b6d_b7a9_4e147f52be11.slice/crio-2fe44a79e88d877164f367e2e2adaa51a7cbf4cb4a5f9de1b846a62442bdfc86 WatchSource:0}: Error finding container 2fe44a79e88d877164f367e2e2adaa51a7cbf4cb4a5f9de1b846a62442bdfc86: Status 404 returned error can't find the container with id 2fe44a79e88d877164f367e2e2adaa51a7cbf4cb4a5f9de1b846a62442bdfc86
Sep 18 20:09:37.994241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:37.994201    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4" event={"ID":"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11","Type":"ContainerStarted","Data":"2fe44a79e88d877164f367e2e2adaa51a7cbf4cb4a5f9de1b846a62442bdfc86"}
Sep 18 20:09:37.996183 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:37.996157    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm" event={"ID":"0ae879f3-7285-44eb-8bac-6c2549860955","Type":"ContainerStarted","Data":"de3195d5a3a53f6659bf5e28a27e04b24e485dd55cf3fbcdaea44661ba12926b"}
Sep 18 20:09:37.996324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:37.996282    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:09:38.027481 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:38.027428    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm" podStartSLOduration=1.54664209 podStartE2EDuration="4.027402854s" totalImagesPullingTime="2.480760764s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:38.023801894 +0000 UTC m=+522.153103846" watchObservedRunningTime="2026-09-18 20:09:38.027402854 +0000 UTC m=+522.156704806"
Sep 18 20:09:38.076502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:09:38.076477    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd260a66a_c621_4346_aaf0_e3b0f5374480.slice/crio-1c5daf78d5f1db586d3e3347ca95c26fef0ecb4522ff7c0a39b35ec8bc992517 WatchSource:0}: Error finding container 1c5daf78d5f1db586d3e3347ca95c26fef0ecb4522ff7c0a39b35ec8bc992517: Status 404 returned error can't find the container with id 1c5daf78d5f1db586d3e3347ca95c26fef0ecb4522ff7c0a39b35ec8bc992517
Sep 18 20:09:38.080226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:38.080208    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"]
Sep 18 20:09:39.002833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:39.002785    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz" event={"ID":"d260a66a-c621-4346-aaf0-e3b0f5374480","Type":"ContainerStarted","Data":"1c5daf78d5f1db586d3e3347ca95c26fef0ecb4522ff7c0a39b35ec8bc992517"}
Sep 18 20:09:41.014498 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:41.014465    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4" event={"ID":"2be4f5eb-ab90-4b6d-b7a9-4e147f52be11","Type":"ContainerStarted","Data":"c5a14fd6e24fd92b8fc318e0e83b95f0a461ff81ac525bbd2c09bdfcf3c42196"}
Sep 18 20:09:41.014890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:41.014551    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:09:41.032833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:41.032789    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4" podStartSLOduration=2.102920725 podStartE2EDuration="5.032776108s" totalImagesPullingTime="2.929855383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:41.031352174 +0000 UTC m=+525.160654126" watchObservedRunningTime="2026-09-18 20:09:41.032776108 +0000 UTC m=+525.162078059"
Sep 18 20:09:43.025344 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:43.025304    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz" event={"ID":"d260a66a-c621-4346-aaf0-e3b0f5374480","Type":"ContainerStarted","Data":"b53a6ecb2f6933a148465205ebcc4abff2311f18e8f9c730dc5234249edd0d4d"}
Sep 18 20:09:43.025698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:43.025447    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:09:43.042417 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:43.042366    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz" podStartSLOduration=3.191053759 podStartE2EDuration="7.042346854s" totalImagesPullingTime="3.851293095s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:09:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:09:43.040201119 +0000 UTC m=+527.169503070" watchObservedRunningTime="2026-09-18 20:09:43.042346854 +0000 UTC m=+527.171648810"
Sep 18 20:09:49.005511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:09:49.005480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-76d6fbff56-zrxqm"
Sep 18 20:10:12.024568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:12.024536    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-2ctb4"
Sep 18 20:10:14.032405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:14.032376    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-728vz"
Sep 18 20:10:40.973610 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.973575    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-84b44d47-7ckhq"]
Sep 18 20:10:40.976143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.976119    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:40.980648 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.980624    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
Sep 18 20:10:40.980648 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.980624    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
Sep 18 20:10:40.981150 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.981133    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
Sep 18 20:10:40.981689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.981673    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
Sep 18 20:10:40.981762 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.981682    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-j65gf\""
Sep 18 20:10:40.984114 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.984094    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
Sep 18 20:10:40.984238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.984220    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
Sep 18 20:10:40.984631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.984616    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
Sep 18 20:10:40.998106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:40.998069    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
Sep 18 20:10:41.012593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.012562    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b44d47-7ckhq"]
Sep 18 20:10:41.023191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-trusted-ca-bundle\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023195    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-oauth-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023227    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023319    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-oauth-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023522 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023408    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-service-ca\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.023562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.023518    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7jx5\" (UniqueName: \"kubernetes.io/projected/79cdcc90-94be-419e-ac93-1c4122a6e0cb-kube-api-access-m7jx5\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.124867 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.124824    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.124867 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.124868    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-oauth-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.124896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.124945    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-service-ca\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7jx5\" (UniqueName: \"kubernetes.io/projected/79cdcc90-94be-419e-ac93-1c4122a6e0cb-kube-api-access-m7jx5\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125089    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-trusted-ca-bundle\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125115    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-oauth-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125861 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125832    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-oauth-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125861 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125849    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-service-ca\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.125861 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.125832    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.126154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.126044    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/79cdcc90-94be-419e-ac93-1c4122a6e0cb-trusted-ca-bundle\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.127575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.127547    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-serving-cert\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.127674 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.127616    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/79cdcc90-94be-419e-ac93-1c4122a6e0cb-console-oauth-config\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.133644 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.133620    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7jx5\" (UniqueName: \"kubernetes.io/projected/79cdcc90-94be-419e-ac93-1c4122a6e0cb-kube-api-access-m7jx5\") pod \"console-84b44d47-7ckhq\" (UID: \"79cdcc90-94be-419e-ac93-1c4122a6e0cb\") " pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.286044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.285929    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:41.560204 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:41.560128    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-84b44d47-7ckhq"]
Sep 18 20:10:41.563593 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:10:41.563570    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79cdcc90_94be_419e_ac93_1c4122a6e0cb.slice/crio-4e05e45e36ce4a4c6b552a85f5306b9f2b741a612c05f756003a077145fb111b WatchSource:0}: Error finding container 4e05e45e36ce4a4c6b552a85f5306b9f2b741a612c05f756003a077145fb111b: Status 404 returned error can't find the container with id 4e05e45e36ce4a4c6b552a85f5306b9f2b741a612c05f756003a077145fb111b
Sep 18 20:10:42.291611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:42.291578    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b44d47-7ckhq" event={"ID":"79cdcc90-94be-419e-ac93-1c4122a6e0cb","Type":"ContainerStarted","Data":"e1f1977fab8073f14fb3a6b1e55e45cb50e2ae68de3ddd22e08a1dad3f4b5387"}
Sep 18 20:10:42.292015 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:42.291630    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-84b44d47-7ckhq" event={"ID":"79cdcc90-94be-419e-ac93-1c4122a6e0cb","Type":"ContainerStarted","Data":"4e05e45e36ce4a4c6b552a85f5306b9f2b741a612c05f756003a077145fb111b"}
Sep 18 20:10:42.311415 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:42.311348    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-84b44d47-7ckhq" podStartSLOduration=2.311334738 podStartE2EDuration="2.311334738s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:10:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:10:42.310025996 +0000 UTC m=+586.439327947" watchObservedRunningTime="2026-09-18 20:10:42.311334738 +0000 UTC m=+586.440636715"
Sep 18 20:10:48.550525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.550489    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"]
Sep 18 20:10:48.553206 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.553183    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.555752 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.555727    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 18 20:10:48.555887 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.555731    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tkd9r\""
Sep 18 20:10:48.555887 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.555731    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 18 20:10:48.564458 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.564430    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"]
Sep 18 20:10:48.691861 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.691823    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.692036 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.691907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.793256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.793210    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.793452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.793283    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.793715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.793694    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.808659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.808578    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf\") pod \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:48.864781 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:48.864735    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:49.148706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:49.148520    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"]
Sep 18 20:10:49.152034 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:10:49.152005    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a9857d1_44fe_471f_83d4_64e378b03b29.slice/crio-a50439cfa4cd8f23e55093cf49de50472244b816e06ad3fe7f45fba6b9146bdb WatchSource:0}: Error finding container a50439cfa4cd8f23e55093cf49de50472244b816e06ad3fe7f45fba6b9146bdb: Status 404 returned error can't find the container with id a50439cfa4cd8f23e55093cf49de50472244b816e06ad3fe7f45fba6b9146bdb
Sep 18 20:10:49.334848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:49.334812    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc" event={"ID":"5a9857d1-44fe-471f-83d4-64e378b03b29","Type":"ContainerStarted","Data":"a50439cfa4cd8f23e55093cf49de50472244b816e06ad3fe7f45fba6b9146bdb"}
Sep 18 20:10:51.286532 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:51.286488    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:51.287348 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:51.286567    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:51.293144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:51.293107    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:51.353821 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:51.353785    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-84b44d47-7ckhq"
Sep 18 20:10:52.808407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.808371    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"]
Sep 18 20:10:52.811226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.811204    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:52.813957 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.813930    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 18 20:10:52.814133 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.813937    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-jchc4\""
Sep 18 20:10:52.825380 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.825355    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"]
Sep 18 20:10:52.937575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:52.937486    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dvc57\" (UniqueName: \"kubernetes.io/projected/4c098713-8210-44ee-aefd-5d523861830b-kube-api-access-dvc57\") pod \"dns-operator-controller-manager-7b84cd7c9c-4s2rd\" (UID: \"4c098713-8210-44ee-aefd-5d523861830b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:53.039129 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.039030    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dvc57\" (UniqueName: \"kubernetes.io/projected/4c098713-8210-44ee-aefd-5d523861830b-kube-api-access-dvc57\") pod \"dns-operator-controller-manager-7b84cd7c9c-4s2rd\" (UID: \"4c098713-8210-44ee-aefd-5d523861830b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:53.048453 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.048383    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dvc57\" (UniqueName: \"kubernetes.io/projected/4c098713-8210-44ee-aefd-5d523861830b-kube-api-access-dvc57\") pod \"dns-operator-controller-manager-7b84cd7c9c-4s2rd\" (UID: \"4c098713-8210-44ee-aefd-5d523861830b\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:53.134557 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.134221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:53.361210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.361168    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc" event={"ID":"5a9857d1-44fe-471f-83d4-64e378b03b29","Type":"ContainerStarted","Data":"d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0"}
Sep 18 20:10:53.361408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.361278    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:10:53.379101 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.379035    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc" podStartSLOduration=1.287064559 podStartE2EDuration="5.379018184s" totalImagesPullingTime="4.091953625s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:10:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:10:53.377247708 +0000 UTC m=+597.506549658" watchObservedRunningTime="2026-09-18 20:10:53.379018184 +0000 UTC m=+597.508320136"
Sep 18 20:10:53.469243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:53.469212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"]
Sep 18 20:10:53.472411 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:10:53.472381    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4c098713_8210_44ee_aefd_5d523861830b.slice/crio-643579f1fdb88abd0283932d613bb1d6fd9f4ab2bafbcd278a5bfd34170fe8eb WatchSource:0}: Error finding container 643579f1fdb88abd0283932d613bb1d6fd9f4ab2bafbcd278a5bfd34170fe8eb: Status 404 returned error can't find the container with id 643579f1fdb88abd0283932d613bb1d6fd9f4ab2bafbcd278a5bfd34170fe8eb
Sep 18 20:10:54.367351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:54.367313    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd" event={"ID":"4c098713-8210-44ee-aefd-5d523861830b","Type":"ContainerStarted","Data":"643579f1fdb88abd0283932d613bb1d6fd9f4ab2bafbcd278a5bfd34170fe8eb"}
Sep 18 20:10:55.373686 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:55.373648    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd" event={"ID":"4c098713-8210-44ee-aefd-5d523861830b","Type":"ContainerStarted","Data":"82093d83c1490d99ea1162751600c69b190cee1aa7585f9ff91fd512c87b0f54"}
Sep 18 20:10:55.374103 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:55.373711    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:10:55.391579 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:55.391535    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd" podStartSLOduration=1.60053399 podStartE2EDuration="3.391519964s" totalImagesPullingTime="1.790985974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:10:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:10:55.39101559 +0000 UTC m=+599.520317540" watchObservedRunningTime="2026-09-18 20:10:55.391519964 +0000 UTC m=+599.520821915"
Sep 18 20:10:58.036759 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:58.036733    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:10:58.037585 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:10:58.037563    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:11:04.369872 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:04.369838    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:06.263974 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.263943    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"]
Sep 18 20:11:06.266351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.266333    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.281421 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.281391    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"]
Sep 18 20:11:06.371945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.371900    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"]
Sep 18 20:11:06.372243 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.372218    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-p4726], unattached volumes=[], failed to process volumes=[]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg" podUID="29d1ae42-1417-4491-8643-031f6f28d03c"
Sep 18 20:11:06.380619 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.380591    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-4s2rd"
Sep 18 20:11:06.421390 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.421355    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"]
Sep 18 20:11:06.423908 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.423885    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.427160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.427133    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.440124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.440096    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"]
Sep 18 20:11:06.443107 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.443081    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.461776 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.461741    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"]
Sep 18 20:11:06.461949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.461855    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.461949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.461931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4726\" (UniqueName: \"kubernetes.io/projected/29d1ae42-1417-4491-8643-031f6f28d03c-kube-api-access-p4726\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.462107 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.462083    2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[extensions-socket-volume kube-api-access-tgvcp], unattached volumes=[], failed to process volumes=[extensions-socket-volume kube-api-access-tgvcp]: context canceled" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95" podUID="2839ba57-36c2-4134-b86f-697199661406"
Sep 18 20:11:06.467801 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.467770    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"]
Sep 18 20:11:06.483567 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.483531    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"]
Sep 18 20:11:06.483754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.483714    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" containerName="manager" containerID="cri-o://d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0" gracePeriod=2
Sep 18 20:11:06.487661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.487632    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:11:06.490332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.490312    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.496452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.496428    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"]
Sep 18 20:11:06.519138 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.519045    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:11:06.521804 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.521772    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"]
Sep 18 20:11:06.524214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.524186    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:06.524601 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.524584    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" containerName="manager"
Sep 18 20:11:06.524601 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.524602    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" containerName="manager"
Sep 18 20:11:06.524781 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.524658    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" containerName="manager"
Sep 18 20:11:06.526920 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.526898    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.529792 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.529761    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"]
Sep 18 20:11:06.538585 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.538544    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:06.538790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.538768    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:06.562194 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.562155    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:06.562419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.562401    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.562465 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.562451    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.562513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.562472    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgvcp\" (UniqueName: \"kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.562557 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.562537    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4726\" (UniqueName: \"kubernetes.io/projected/29d1ae42-1417-4491-8643-031f6f28d03c-kube-api-access-p4726\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.562903 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.562878    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") " pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:06.567538 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.567515    2566 projected.go:196] Error preparing data for projected volume kube-api-access-p4726 for pod kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg: failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-pm4kg" not found
Sep 18 20:11:06.567655 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.567575    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29d1ae42-1417-4491-8643-031f6f28d03c-kube-api-access-p4726 podName:29d1ae42-1417-4491-8643-031f6f28d03c nodeName:}" failed. No retries permitted until 2026-09-18 20:11:07.067559679 +0000 UTC m=+611.196861613 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-p4726" (UniqueName: "kubernetes.io/projected/29d1ae42-1417-4491-8643-031f6f28d03c-kube-api-access-p4726") pod "kuadrant-operator-controller-manager-7978cbdfbc-pm4kg" (UID: "29d1ae42-1417-4491-8643-031f6f28d03c") : failed to fetch token: pod "kuadrant-operator-controller-manager-7978cbdfbc-pm4kg" not found
Sep 18 20:11:06.663698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.663662    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume\") pod \"29d1ae42-1417-4491-8643-031f6f28d03c\" (UID: \"29d1ae42-1417-4491-8643-031f6f28d03c\") "
Sep 18 20:11:06.663901 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.663878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.663975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.663952    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.663975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.663957    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "29d1ae42-1417-4491-8643-031f6f28d03c" (UID: "29d1ae42-1417-4491-8643-031f6f28d03c"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:11:06.664139 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664014    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgvcp\" (UniqueName: \"kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.664139 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664106    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.664241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dlklw\" (UniqueName: \"kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.664288 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664267    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frsjs\" (UniqueName: \"kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.664323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664297    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:06.664407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664392    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/29d1ae42-1417-4491-8643-031f6f28d03c-extensions-socket-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:06.664444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.664415    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4726\" (UniqueName: \"kubernetes.io/projected/29d1ae42-1417-4491-8643-031f6f28d03c-kube-api-access-p4726\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:06.666988 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.666965    2566 projected.go:196] Error preparing data for projected volume kube-api-access-tgvcp for pod kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95: failed to fetch token: pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" not found
Sep 18 20:11:06.667139 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.667047    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp podName:2839ba57-36c2-4134-b86f-697199661406 nodeName:}" failed. No retries permitted until 2026-09-18 20:11:07.167025107 +0000 UTC m=+611.296327039 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-tgvcp" (UniqueName: "kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp") pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" (UID: "2839ba57-36c2-4134-b86f-697199661406") : failed to fetch token: pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" not found
Sep 18 20:11:06.723438 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.723407    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:06.725736 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:06.725709    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:06.765367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765327    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frsjs\" (UniqueName: \"kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.765540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765377    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.765540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765423    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.765610 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dlklw\" (UniqueName: \"kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.765869 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765846    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.765907 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.765849    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.774909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.774843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dlklw\" (UniqueName: \"kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw\") pod \"kuadrant-operator-controller-manager-76f8f686bf-c6w4d\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.774909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.774879    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frsjs\" (UniqueName: \"kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs\") pod \"kuadrant-operator-controller-manager-5769f99d9f-999v9\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.856457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.856415    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:06.864568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.864540    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:06.866569 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.866456    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume\") pod \"5a9857d1-44fe-471f-83d4-64e378b03b29\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") "
Sep 18 20:11:06.866569 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.866494    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf\") pod \"5a9857d1-44fe-471f-83d4-64e378b03b29\" (UID: \"5a9857d1-44fe-471f-83d4-64e378b03b29\") "
Sep 18 20:11:06.866875 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.866852    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "5a9857d1-44fe-471f-83d4-64e378b03b29" (UID: "5a9857d1-44fe-471f-83d4-64e378b03b29"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:11:06.868620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.868594    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf" (OuterVolumeSpecName: "kube-api-access-rkvxf") pod "5a9857d1-44fe-471f-83d4-64e378b03b29" (UID: "5a9857d1-44fe-471f-83d4-64e378b03b29"). InnerVolumeSpecName "kube-api-access-rkvxf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:11:06.967685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.967650    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/5a9857d1-44fe-471f-83d4-64e378b03b29-extensions-socket-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:06.967685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:06.967682    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkvxf\" (UniqueName: \"kubernetes.io/projected/5a9857d1-44fe-471f-83d4-64e378b03b29-kube-api-access-rkvxf\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:07.151531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.151506    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:11:07.153966 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:11:07.153944    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod918a35d0_3c67_4f38_b13a_213a797b6aa2.slice/crio-b2df7770b13d034915ae9aa261e9a15eb2c1a5d60f0f5f7561d12f658e201117 WatchSource:0}: Error finding container b2df7770b13d034915ae9aa261e9a15eb2c1a5d60f0f5f7561d12f658e201117: Status 404 returned error can't find the container with id b2df7770b13d034915ae9aa261e9a15eb2c1a5d60f0f5f7561d12f658e201117
Sep 18 20:11:07.170010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.169978    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgvcp\" (UniqueName: \"kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp\") pod \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:07.173319 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.173292    2566 projected.go:196] Error preparing data for projected volume kube-api-access-tgvcp for pod kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95: failed to fetch token: pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" not found
Sep 18 20:11:07.173438 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.173414    2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp podName:2839ba57-36c2-4134-b86f-697199661406 nodeName:}" failed. No retries permitted until 2026-09-18 20:11:08.173392196 +0000 UTC m=+612.302694135 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-tgvcp" (UniqueName: "kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp") pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" (UID: "2839ba57-36c2-4134-b86f-697199661406") : failed to fetch token: pod "kuadrant-operator-controller-manager-5769f99d9f-mzg95" not found
Sep 18 20:11:07.173640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.173618    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:07.187138 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:11:07.187112    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9dc231ed_789e_4937_b7f5_1b1791b94f7b.slice/crio-ea8b981a18b3cdfbb7102727c1ca66e10057eaf33e05f09c017021c1ffcf2c69 WatchSource:0}: Error finding container ea8b981a18b3cdfbb7102727c1ca66e10057eaf33e05f09c017021c1ffcf2c69: Status 404 returned error can't find the container with id ea8b981a18b3cdfbb7102727c1ca66e10057eaf33e05f09c017021c1ffcf2c69
Sep 18 20:11:07.434176 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.434087    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" event={"ID":"918a35d0-3c67-4f38-b13a-213a797b6aa2","Type":"ContainerStarted","Data":"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"}
Sep 18 20:11:07.434622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.434248    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:07.434622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.434300    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" event={"ID":"918a35d0-3c67-4f38-b13a-213a797b6aa2","Type":"ContainerStarted","Data":"b2df7770b13d034915ae9aa261e9a15eb2c1a5d60f0f5f7561d12f658e201117"}
Sep 18 20:11:07.435515 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.435491    2566 generic.go:356] "Generic (PLEG): container finished" podID="5a9857d1-44fe-471f-83d4-64e378b03b29" containerID="d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0" exitCode=0
Sep 18 20:11:07.435636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.435534    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:07.435636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.435585    2566 scope.go:117] "RemoveContainer" containerID="d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0"
Sep 18 20:11:07.436331 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.436304    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:07.437707 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.437677    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" event={"ID":"9dc231ed-789e-4937-b7f5-1b1791b94f7b","Type":"ContainerStarted","Data":"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"}
Sep 18 20:11:07.437807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.437716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:07.437807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.437732    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" event={"ID":"9dc231ed-789e-4937-b7f5-1b1791b94f7b","Type":"ContainerStarted","Data":"ea8b981a18b3cdfbb7102727c1ca66e10057eaf33e05f09c017021c1ffcf2c69"}
Sep 18 20:11:07.437807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.437716    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:07.437932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.437897    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:07.451543 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.451008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:07.453618 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.453592    2566 scope.go:117] "RemoveContainer" containerID="d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0"
Sep 18 20:11:07.454124 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.454098    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0\": container with ID starting with d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0 not found: ID does not exist" containerID="d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0"
Sep 18 20:11:07.454218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.454136    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0"} err="failed to get container status \"d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0\": rpc error: code = NotFound desc = could not find container \"d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0\": container with ID starting with d87f4add02970a22195670df1e3b4d338c4f32f5562bdad8f72b721e5051a6c0 not found: ID does not exist"
Sep 18 20:11:07.461565 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.461520    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" podStartSLOduration=1.461503672 podStartE2EDuration="1.461503672s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:07.459700121 +0000 UTC m=+611.589002102" watchObservedRunningTime="2026-09-18 20:11:07.461503672 +0000 UTC m=+611.590805705"
Sep 18 20:11:07.463539 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.463510    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="29d1ae42-1417-4491-8643-031f6f28d03c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:07.487583 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.487530    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" podStartSLOduration=1.487512464 podStartE2EDuration="1.487512464s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:07.48586578 +0000 UTC m=+611.615167732" watchObservedRunningTime="2026-09-18 20:11:07.487512464 +0000 UTC m=+611.616814415"
Sep 18 20:11:07.488503 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.488479    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:07.490176 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:07.490155    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="2839ba57-36c2-4134-b86f-697199661406" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:07.572429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.572393    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume\") pod \"2839ba57-36c2-4134-b86f-697199661406\" (UID: \"2839ba57-36c2-4134-b86f-697199661406\") "
Sep 18 20:11:07.572669 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.572642    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2839ba57-36c2-4134-b86f-697199661406" (UID: "2839ba57-36c2-4134-b86f-697199661406"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:11:07.572825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.572808    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tgvcp\" (UniqueName: \"kubernetes.io/projected/2839ba57-36c2-4134-b86f-697199661406-kube-api-access-tgvcp\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:07.572874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:07.572830    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2839ba57-36c2-4134-b86f-697199661406-extensions-socket-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:08.078496 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:08.078455    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="2839ba57-36c2-4134-b86f-697199661406" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:08.080228 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:08.080199    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-7978cbdfbc-pm4kg\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="29d1ae42-1417-4491-8643-031f6f28d03c" pod="kuadrant-system/kuadrant-operator-controller-manager-7978cbdfbc-pm4kg"
Sep 18 20:11:08.080374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:08.080343    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2839ba57-36c2-4134-b86f-697199661406" path="/var/lib/kubelet/pods/2839ba57-36c2-4134-b86f-697199661406/volumes"
Sep 18 20:11:08.080562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:08.080549    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29d1ae42-1417-4491-8643-031f6f28d03c" path="/var/lib/kubelet/pods/29d1ae42-1417-4491-8643-031f6f28d03c/volumes"
Sep 18 20:11:08.080788 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:08.080777    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" path="/var/lib/kubelet/pods/5a9857d1-44fe-471f-83d4-64e378b03b29/volumes"
Sep 18 20:11:08.081911 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:08.081889    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-9wdwc\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="5a9857d1-44fe-471f-83d4-64e378b03b29" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-9wdwc"
Sep 18 20:11:08.445362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:08.445269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:08.449711 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:08.449677    2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-5769f99d9f-mzg95\" is forbidden: User \"system:node:ip-10-0-131-20.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-131-20.ec2.internal' and this object" podUID="2839ba57-36c2-4134-b86f-697199661406" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-mzg95"
Sep 18 20:11:18.447568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.447535    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:11:18.448084 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.447636    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:18.531647 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.531607    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:18.531878 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.531850    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" podUID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" containerName="manager" containerID="cri-o://85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c" gracePeriod=10
Sep 18 20:11:18.807048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.807011    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:18.871040 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.871002    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dlklw\" (UniqueName: \"kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw\") pod \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") "
Sep 18 20:11:18.871240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.871143    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume\") pod \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\" (UID: \"9dc231ed-789e-4937-b7f5-1b1791b94f7b\") "
Sep 18 20:11:18.871516 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.871488    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9dc231ed-789e-4937-b7f5-1b1791b94f7b" (UID: "9dc231ed-789e-4937-b7f5-1b1791b94f7b"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:11:18.873119 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.873095    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw" (OuterVolumeSpecName: "kube-api-access-dlklw") pod "9dc231ed-789e-4937-b7f5-1b1791b94f7b" (UID: "9dc231ed-789e-4937-b7f5-1b1791b94f7b"). InnerVolumeSpecName "kube-api-access-dlklw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:11:18.972374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.972277    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9dc231ed-789e-4937-b7f5-1b1791b94f7b-extensions-socket-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:18.972374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:18.972311    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dlklw\" (UniqueName: \"kubernetes.io/projected/9dc231ed-789e-4937-b7f5-1b1791b94f7b-kube-api-access-dlklw\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:19.502000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.501955    2566 generic.go:356] "Generic (PLEG): container finished" podID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" containerID="85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c" exitCode=0
Sep 18 20:11:19.502444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.502009    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"
Sep 18 20:11:19.502444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.502024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" event={"ID":"9dc231ed-789e-4937-b7f5-1b1791b94f7b","Type":"ContainerDied","Data":"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"}
Sep 18 20:11:19.502444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.502080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d" event={"ID":"9dc231ed-789e-4937-b7f5-1b1791b94f7b","Type":"ContainerDied","Data":"ea8b981a18b3cdfbb7102727c1ca66e10057eaf33e05f09c017021c1ffcf2c69"}
Sep 18 20:11:19.502444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.502103    2566 scope.go:117] "RemoveContainer" containerID="85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"
Sep 18 20:11:19.517424 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.517399    2566 scope.go:117] "RemoveContainer" containerID="85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"
Sep 18 20:11:19.517773 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:19.517750    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c\": container with ID starting with 85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c not found: ID does not exist" containerID="85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"
Sep 18 20:11:19.517820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.517784    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c"} err="failed to get container status \"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c\": rpc error: code = NotFound desc = could not find container \"85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c\": container with ID starting with 85e944468bd6ec60e802e608b00659d7c78364f5f20b884ad9ab8a21898a8b4c not found: ID does not exist"
Sep 18 20:11:19.533990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.533946    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:19.539376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:19.539348    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-c6w4d"]
Sep 18 20:11:20.080879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:20.080843    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" path="/var/lib/kubelet/pods/9dc231ed-789e-4937-b7f5-1b1791b94f7b/volumes"
Sep 18 20:11:34.830828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.830789    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"]
Sep 18 20:11:34.831261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.831187    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" containerName="manager"
Sep 18 20:11:34.831261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.831201    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" containerName="manager"
Sep 18 20:11:34.831347 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.831291    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9dc231ed-789e-4937-b7f5-1b1791b94f7b" containerName="manager"
Sep 18 20:11:34.834452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.834429    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.836890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.836869    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wd2nc\""
Sep 18 20:11:34.845341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.845313    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"]
Sep 18 20:11:34.914490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914516    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914533    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914573    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914624    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914659    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hd5lp\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-kube-api-access-hd5lp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:34.914934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:34.914718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015534    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015552    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015668    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hd5lp\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-kube-api-access-hd5lp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015696    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.015810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015727    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015947    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.015989    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.016203    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.016216    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.016295    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.016365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.016527 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.016508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.018245 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.018221    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.018362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.018306    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.023394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.023360    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.023562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.023543    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hd5lp\" (UniqueName: \"kubernetes.io/projected/7180073a-a6e8-4fa2-9bce-cba93c382bb1-kube-api-access-hd5lp\") pod \"maas-default-gateway-openshift-default-78bb78ff78-tt5l7\" (UID: \"7180073a-a6e8-4fa2-9bce-cba93c382bb1\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.145796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.145695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:35.420970 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.420797    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"]
Sep 18 20:11:35.423844 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:11:35.423811    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7180073a_a6e8_4fa2_9bce_cba93c382bb1.slice/crio-dc62b753fb8502b358aa5e1e2dadac59febb0682fafdc32d24891d2b29fbe3db WatchSource:0}: Error finding container dc62b753fb8502b358aa5e1e2dadac59febb0682fafdc32d24891d2b29fbe3db: Status 404 returned error can't find the container with id dc62b753fb8502b358aa5e1e2dadac59febb0682fafdc32d24891d2b29fbe3db
Sep 18 20:11:35.426007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.425978    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:11:35.426104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.426051    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:11:35.426104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.426101    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:11:35.588113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.588046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" event={"ID":"7180073a-a6e8-4fa2-9bce-cba93c382bb1","Type":"ContainerStarted","Data":"8dd71132a077194b19d3e570c668aeda33bd0ec2afbdf2a786debab15a076cdd"}
Sep 18 20:11:35.588422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.588144    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" event={"ID":"7180073a-a6e8-4fa2-9bce-cba93c382bb1","Type":"ContainerStarted","Data":"dc62b753fb8502b358aa5e1e2dadac59febb0682fafdc32d24891d2b29fbe3db"}
Sep 18 20:11:35.607341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:35.607286    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podStartSLOduration=1.607267966 podStartE2EDuration="1.607267966s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:35.604670919 +0000 UTC m=+639.733972870" watchObservedRunningTime="2026-09-18 20:11:35.607267966 +0000 UTC m=+639.736569917"
Sep 18 20:11:36.146535 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:36.146492    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:36.151817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:36.151790    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:36.593325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:36.593295    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:36.594316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:36.594295    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:11:37.389771 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.389732    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:37.392682 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.392655    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:37.394814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.394792    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2nff\""
Sep 18 20:11:37.401412 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.401387    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:37.542667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.542626    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n4m4l\" (UniqueName: \"kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l\") pod \"authorino-7bcfd8f9cd-r6sjt\" (UID: \"e92b7a62-7d85-4951-8136-5679ce1e8fc8\") " pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:37.643740 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.643647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n4m4l\" (UniqueName: \"kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l\") pod \"authorino-7bcfd8f9cd-r6sjt\" (UID: \"e92b7a62-7d85-4951-8136-5679ce1e8fc8\") " pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:37.651672 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.651643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n4m4l\" (UniqueName: \"kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l\") pod \"authorino-7bcfd8f9cd-r6sjt\" (UID: \"e92b7a62-7d85-4951-8136-5679ce1e8fc8\") " pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:37.703569 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.703530    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:37.973521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.973487    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:37.977730 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:11:37.977701    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode92b7a62_7d85_4951_8136_5679ce1e8fc8.slice/crio-4fc1b43a7124ed3da902eb98abbe8df0faf7a676640cae20c1cf653727cec5b9 WatchSource:0}: Error finding container 4fc1b43a7124ed3da902eb98abbe8df0faf7a676640cae20c1cf653727cec5b9: Status 404 returned error can't find the container with id 4fc1b43a7124ed3da902eb98abbe8df0faf7a676640cae20c1cf653727cec5b9
Sep 18 20:11:37.979318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:37.979295    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:11:38.605641 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:38.605597    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" event={"ID":"e92b7a62-7d85-4951-8136-5679ce1e8fc8","Type":"ContainerStarted","Data":"4fc1b43a7124ed3da902eb98abbe8df0faf7a676640cae20c1cf653727cec5b9"}
Sep 18 20:11:39.612751 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:39.612709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" event={"ID":"e92b7a62-7d85-4951-8136-5679ce1e8fc8","Type":"ContainerStarted","Data":"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"}
Sep 18 20:11:39.628159 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:39.628106    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" podStartSLOduration=1.142152079 podStartE2EDuration="2.628088056s" totalImagesPullingTime="1.485935977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:11:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:11:39.626090248 +0000 UTC m=+643.755392202" watchObservedRunningTime="2026-09-18 20:11:39.628088056 +0000 UTC m=+643.757390007"
Sep 18 20:11:41.008966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:41.008924    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:41.622578 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:41.622539    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" podUID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" containerName="authorino" containerID="cri-o://02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e" gracePeriod=30
Sep 18 20:11:41.874790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:41.874734    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:41.986293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:41.986262    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n4m4l\" (UniqueName: \"kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l\") pod \"e92b7a62-7d85-4951-8136-5679ce1e8fc8\" (UID: \"e92b7a62-7d85-4951-8136-5679ce1e8fc8\") "
Sep 18 20:11:41.988330 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:41.988307    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l" (OuterVolumeSpecName: "kube-api-access-n4m4l") pod "e92b7a62-7d85-4951-8136-5679ce1e8fc8" (UID: "e92b7a62-7d85-4951-8136-5679ce1e8fc8"). InnerVolumeSpecName "kube-api-access-n4m4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:11:42.087121 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.086948    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n4m4l\" (UniqueName: \"kubernetes.io/projected/e92b7a62-7d85-4951-8136-5679ce1e8fc8-kube-api-access-n4m4l\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:11:42.635609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.635570    2566 generic.go:356] "Generic (PLEG): container finished" podID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" containerID="02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e" exitCode=0
Sep 18 20:11:42.635819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.635613    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt"
Sep 18 20:11:42.635819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.635657    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" event={"ID":"e92b7a62-7d85-4951-8136-5679ce1e8fc8","Type":"ContainerDied","Data":"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"}
Sep 18 20:11:42.635819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.635698    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-r6sjt" event={"ID":"e92b7a62-7d85-4951-8136-5679ce1e8fc8","Type":"ContainerDied","Data":"4fc1b43a7124ed3da902eb98abbe8df0faf7a676640cae20c1cf653727cec5b9"}
Sep 18 20:11:42.635819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.635720    2566 scope.go:117] "RemoveContainer" containerID="02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"
Sep 18 20:11:42.644376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.644358    2566 scope.go:117] "RemoveContainer" containerID="02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"
Sep 18 20:11:42.644697 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:11:42.644677    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e\": container with ID starting with 02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e not found: ID does not exist" containerID="02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"
Sep 18 20:11:42.644771 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.644708    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e"} err="failed to get container status \"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e\": rpc error: code = NotFound desc = could not find container \"02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e\": container with ID starting with 02e94870843cab0bb74a76d7c9efcaaed1a29360bab4a88ef9de4fb001f8745e not found: ID does not exist"
Sep 18 20:11:42.650887 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.650861    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:42.655189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:42.655164    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-r6sjt"]
Sep 18 20:11:44.081348 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:11:44.081313    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" path="/var/lib/kubelet/pods/e92b7a62-7d85-4951-8136-5679ce1e8fc8/volumes"
Sep 18 20:12:10.368774 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.368739    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:10.369203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.369137    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" containerName="authorino"
Sep 18 20:12:10.369203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.369150    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" containerName="authorino"
Sep 18 20:12:10.369290 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.369223    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e92b7a62-7d85-4951-8136-5679ce1e8fc8" containerName="authorino"
Sep 18 20:12:10.373716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.373695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:10.376040 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.376009    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-c2nff\""
Sep 18 20:12:10.378559 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.378536    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:10.447853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.447819    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc2tw\" (UniqueName: \"kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw\") pod \"authorino-557f876f69-9gphv\" (UID: \"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23\") " pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:10.548684 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.548647    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc2tw\" (UniqueName: \"kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw\") pod \"authorino-557f876f69-9gphv\" (UID: \"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23\") " pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:10.556105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.556045    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc2tw\" (UniqueName: \"kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw\") pod \"authorino-557f876f69-9gphv\" (UID: \"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23\") " pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:10.610799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.610757    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:10.611105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.611088    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:10.635587 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.635558    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:12:10.639033 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.639013    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.641324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.641303    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 18 20:12:10.648276 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.648250    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:12:10.749901 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.749865    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.750111 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.749919    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m26zd\" (UniqueName: \"kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.850553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.850514    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.850738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.850602    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m26zd\" (UniqueName: \"kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.853007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.852986    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.858699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.858679    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m26zd\" (UniqueName: \"kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd\") pod \"authorino-d96447cdd-kb9wl\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") " pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:10.882509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.882488    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:10.884888 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:12:10.884859    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ab65545_12d7_4e2a_8d5b_605fc8c4cd23.slice/crio-20b0336520049987b6e9bc02ba5aef125e26bde1f26a543d19c70df90bfc0a62 WatchSource:0}: Error finding container 20b0336520049987b6e9bc02ba5aef125e26bde1f26a543d19c70df90bfc0a62: Status 404 returned error can't find the container with id 20b0336520049987b6e9bc02ba5aef125e26bde1f26a543d19c70df90bfc0a62
Sep 18 20:12:10.950307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:10.950277    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:12:11.242175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.242149    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:12:11.244412 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:12:11.244382    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda6027356_c780_4fef_90bd_f9d9674e7bef.slice/crio-f1fb0751dfaf4b4980dc5271230ebaad30003c45b9a6912a74e8a1112e451fb0 WatchSource:0}: Error finding container f1fb0751dfaf4b4980dc5271230ebaad30003c45b9a6912a74e8a1112e451fb0: Status 404 returned error can't find the container with id f1fb0751dfaf4b4980dc5271230ebaad30003c45b9a6912a74e8a1112e451fb0
Sep 18 20:12:11.788612 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.788580    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d96447cdd-kb9wl" event={"ID":"a6027356-c780-4fef-90bd-f9d9674e7bef","Type":"ContainerStarted","Data":"f1fb0751dfaf4b4980dc5271230ebaad30003c45b9a6912a74e8a1112e451fb0"}
Sep 18 20:12:11.790405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.790379    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-9gphv" event={"ID":"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23","Type":"ContainerStarted","Data":"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"}
Sep 18 20:12:11.790534 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.790412    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-9gphv" event={"ID":"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23","Type":"ContainerStarted","Data":"20b0336520049987b6e9bc02ba5aef125e26bde1f26a543d19c70df90bfc0a62"}
Sep 18 20:12:11.790534 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.790426    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-557f876f69-9gphv" podUID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" containerName="authorino" containerID="cri-o://94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62" gracePeriod=30
Sep 18 20:12:11.809117 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:11.809046    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-557f876f69-9gphv" podStartSLOduration=1.002972666 podStartE2EDuration="1.809021303s" totalImagesPullingTime="806.048637ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:12:11.807412131 +0000 UTC m=+675.936714082" watchObservedRunningTime="2026-09-18 20:12:11.809021303 +0000 UTC m=+675.938323254"
Sep 18 20:12:12.041461 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.041424    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:12.163877 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.163843    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bc2tw\" (UniqueName: \"kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw\") pod \"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23\" (UID: \"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23\") "
Sep 18 20:12:12.165939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.165918    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw" (OuterVolumeSpecName: "kube-api-access-bc2tw") pod "9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" (UID: "9ab65545-12d7-4e2a-8d5b-605fc8c4cd23"). InnerVolumeSpecName "kube-api-access-bc2tw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:12:12.264817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.264790    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc2tw\" (UniqueName: \"kubernetes.io/projected/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23-kube-api-access-bc2tw\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:12:12.797015 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.796978    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d96447cdd-kb9wl" event={"ID":"a6027356-c780-4fef-90bd-f9d9674e7bef","Type":"ContainerStarted","Data":"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"}
Sep 18 20:12:12.798438 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.798411    2566 generic.go:356] "Generic (PLEG): container finished" podID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" containerID="94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62" exitCode=0
Sep 18 20:12:12.798572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.798456    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-557f876f69-9gphv"
Sep 18 20:12:12.798572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.798463    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-9gphv" event={"ID":"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23","Type":"ContainerDied","Data":"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"}
Sep 18 20:12:12.798572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.798488    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-557f876f69-9gphv" event={"ID":"9ab65545-12d7-4e2a-8d5b-605fc8c4cd23","Type":"ContainerDied","Data":"20b0336520049987b6e9bc02ba5aef125e26bde1f26a543d19c70df90bfc0a62"}
Sep 18 20:12:12.798572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.798505    2566 scope.go:117] "RemoveContainer" containerID="94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"
Sep 18 20:12:12.808650 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.808227    2566 scope.go:117] "RemoveContainer" containerID="94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"
Sep 18 20:12:12.809799 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:12:12.808882    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62\": container with ID starting with 94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62 not found: ID does not exist" containerID="94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"
Sep 18 20:12:12.809799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.808919    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62"} err="failed to get container status \"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62\": rpc error: code = NotFound desc = could not find container \"94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62\": container with ID starting with 94a6a9697a69b1cb5ea73fe2002a03ab437cf5ece8a4f63d94c4ef3bd0cd7b62 not found: ID does not exist"
Sep 18 20:12:12.812698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.812633    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-d96447cdd-kb9wl" podStartSLOduration=2.25542585 podStartE2EDuration="2.812617616s" totalImagesPullingTime="557.191766ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:12:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:12:12.810611181 +0000 UTC m=+676.939913131" watchObservedRunningTime="2026-09-18 20:12:12.812617616 +0000 UTC m=+676.941919566"
Sep 18 20:12:12.825252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.825227    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:12.829301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:12.829278    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-557f876f69-9gphv"]
Sep 18 20:12:14.080646 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:14.080615    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" path="/var/lib/kubelet/pods/9ab65545-12d7-4e2a-8d5b-605fc8c4cd23/volumes"
Sep 18 20:12:55.632597 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.632564    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:12:55.633036 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.632929    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" containerName="authorino"
Sep 18 20:12:55.633036 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.632940    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" containerName="authorino"
Sep 18 20:12:55.633036 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.633000    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9ab65545-12d7-4e2a-8d5b-605fc8c4cd23" containerName="authorino"
Sep 18 20:12:55.635374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.635358    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.637491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.637470    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 20:12:55.637623 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.637501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-dmsxp\""
Sep 18 20:12:55.645239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.645218    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:12:55.762967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.762931    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2xcg4\" (UniqueName: \"kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.762967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.762971    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.865478 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.864426    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2xcg4\" (UniqueName: \"kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.865478 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.864488    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.865478 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.865402    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.873360 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.873335    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2xcg4\" (UniqueName: \"kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4\") pod \"payload-pre-processing-59fbc47bf8-bhm7v\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") " pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:55.946785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:55.946701    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:56.237106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:56.237079    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:12:56.239247 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:12:56.239213    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76d9426f_4053_4155_8700_cb7dc1aa4a6c.slice/crio-3315d8ec6592a511a0c18588d2e3018787bbfa049ba97a7a231feeabd53f19ee WatchSource:0}: Error finding container 3315d8ec6592a511a0c18588d2e3018787bbfa049ba97a7a231feeabd53f19ee: Status 404 returned error can't find the container with id 3315d8ec6592a511a0c18588d2e3018787bbfa049ba97a7a231feeabd53f19ee
Sep 18 20:12:57.019803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:57.019766    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" event={"ID":"76d9426f-4053-4155-8700-cb7dc1aa4a6c","Type":"ContainerStarted","Data":"3315d8ec6592a511a0c18588d2e3018787bbfa049ba97a7a231feeabd53f19ee"}
Sep 18 20:12:58.026839 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:58.026803    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" event={"ID":"76d9426f-4053-4155-8700-cb7dc1aa4a6c","Type":"ContainerStarted","Data":"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"}
Sep 18 20:12:58.027244 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:58.026910    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:12:58.042639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:12:58.042586    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" podStartSLOduration=1.672659458 podStartE2EDuration="3.042535126s" totalImagesPullingTime="1.369875668s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:12:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:12:58.041077143 +0000 UTC m=+722.170379092" watchObservedRunningTime="2026-09-18 20:12:58.042535126 +0000 UTC m=+722.171837078"
Sep 18 20:13:06.069907 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.069873    2566 generic.go:356] "Generic (PLEG): container finished" podID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerID="8dd71132a077194b19d3e570c668aeda33bd0ec2afbdf2a786debab15a076cdd" exitCode=0
Sep 18 20:13:06.070351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.069947    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" event={"ID":"7180073a-a6e8-4fa2-9bce-cba93c382bb1","Type":"ContainerDied","Data":"8dd71132a077194b19d3e570c668aeda33bd0ec2afbdf2a786debab15a076cdd"}
Sep 18 20:13:06.070440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.070419    2566 scope.go:117] "RemoveContainer" containerID="8dd71132a077194b19d3e570c668aeda33bd0ec2afbdf2a786debab15a076cdd"
Sep 18 20:13:06.071842 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.071783    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:13:06.071965 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.071871    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:13:06.071965 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.071922    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:13:06.593178 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:06.593144    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:13:07.076583 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:07.076548    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" event={"ID":"7180073a-a6e8-4fa2-9bce-cba93c382bb1","Type":"ContainerStarted","Data":"06af8ba5463fedb0c19fbfc701014cfaa72b10e18d64933fba6158b0116d6c16"}
Sep 18 20:13:07.146606 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:07.146567    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:13:07.147748 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:07.147714    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tt5l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 20:13:07.147857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:07.147774    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podUID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 20:13:08.154496 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:08.154460    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tt5l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 20:13:08.155088 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:08.155030    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podUID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 20:13:09.034016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:09.033987    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:13:09.151082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:09.146872    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tt5l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 20:13:09.151082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:09.146968    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podUID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 20:13:10.157082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:10.152397    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tt5l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 20:13:10.157082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:10.152472    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podUID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 20:13:11.147081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:11.147023    2566 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-tt5l7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused" start-of-body=
Sep 18 20:13:11.147362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:11.147335    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7" podUID="7180073a-a6e8-4fa2-9bce-cba93c382bb1" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.36:15021/healthz/ready\": dial tcp 10.132.0.36:15021: connect: connection refused"
Sep 18 20:13:12.150553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:12.150524    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:13:12.150998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:12.150798    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:13:12.151470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:12.151454    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-tt5l7"
Sep 18 20:13:17.504351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.504319    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"]
Sep 18 20:13:17.507004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.506985    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.510372 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.510349    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 18 20:13:17.510471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.510401    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 18 20:13:17.510471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.510427    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-m7b4l\""
Sep 18 20:13:17.510471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.510448    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
Sep 18 20:13:17.518908 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.518886    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"]
Sep 18 20:13:17.565458 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h59d\" (UniqueName: \"kubernetes.io/projected/63b8daaf-21d1-4756-afba-6065e093a571-kube-api-access-2h59d\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.565598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565465    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b8daaf-21d1-4756-afba-6065e093a571-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.565598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.565694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.565694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565642    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.565694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.565687    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.666395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666352    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.666658 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666638    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.666803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666779    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.666961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666841    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.666961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666886    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.667167 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.667031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.667227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.666896    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h59d\" (UniqueName: \"kubernetes.io/projected/63b8daaf-21d1-4756-afba-6065e093a571-kube-api-access-2h59d\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.667337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.667228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b8daaf-21d1-4756-afba-6065e093a571-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.667337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.667252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.669316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.669291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/63b8daaf-21d1-4756-afba-6065e093a571-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.669643 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.669627    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/63b8daaf-21d1-4756-afba-6065e093a571-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.674551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.674534    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h59d\" (UniqueName: \"kubernetes.io/projected/63b8daaf-21d1-4756-afba-6065e093a571-kube-api-access-2h59d\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc\" (UID: \"63b8daaf-21d1-4756-afba-6065e093a571\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:17.817673 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:17.817587    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:18.090010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:18.089928    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"]
Sep 18 20:13:18.093097 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:13:18.093049    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod63b8daaf_21d1_4756_afba_6065e093a571.slice/crio-965bc257c7c8e7a2da75553490b222049f9fd761c403278609f1e2333904a983 WatchSource:0}: Error finding container 965bc257c7c8e7a2da75553490b222049f9fd761c403278609f1e2333904a983: Status 404 returned error can't find the container with id 965bc257c7c8e7a2da75553490b222049f9fd761c403278609f1e2333904a983
Sep 18 20:13:18.137080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:18.137028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc" event={"ID":"63b8daaf-21d1-4756-afba-6065e093a571","Type":"ContainerStarted","Data":"965bc257c7c8e7a2da75553490b222049f9fd761c403278609f1e2333904a983"}
Sep 18 20:13:20.148962 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:20.148928    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc" event={"ID":"63b8daaf-21d1-4756-afba-6065e093a571","Type":"ContainerStarted","Data":"f9af413248fd3a4288a6a2abcd394a86cee733b6251fd63e201a2d99e4c66bff"}
Sep 18 20:13:20.165011 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:20.164960    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc" podStartSLOduration=2.118221579 podStartE2EDuration="3.164940421s" totalImagesPullingTime="1.046718842s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:13:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:13:20.163868682 +0000 UTC m=+744.293170659" watchObservedRunningTime="2026-09-18 20:13:20.164940421 +0000 UTC m=+744.294242377"
Sep 18 20:13:23.993636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:23.993596    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"]
Sep 18 20:13:23.996457 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:23.996436    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:23.998732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:23.998712    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\""
Sep 18 20:13:24.005468 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.005443    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"]
Sep 18 20:13:24.018904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.018877    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.019151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.019128    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.019256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.019176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.019256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.019204    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nlrf\" (UniqueName: \"kubernetes.io/projected/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-kube-api-access-2nlrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.019256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.019242    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.019417 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.019269    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120517    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120554    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2nlrf\" (UniqueName: \"kubernetes.io/projected/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-kube-api-access-2nlrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120583    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120606    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120738    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.120991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120968    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.121049 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.120993    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.121049 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.121041    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.122714 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.122692    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.123160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.123142    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.127553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.127532    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nlrf\" (UniqueName: \"kubernetes.io/projected/488db7c3-8e3a-424e-bc1e-6c98c08a9e88-kube-api-access-2nlrf\") pod \"e2e-distinct-simulated-kserve-59975d69f9-84bjl\" (UID: \"488db7c3-8e3a-424e-bc1e-6c98c08a9e88\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.308120 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.308025    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:24.579604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:24.579580    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"]
Sep 18 20:13:24.581502 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:13:24.581470    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod488db7c3_8e3a_424e_bc1e_6c98c08a9e88.slice/crio-50121931fa8e646b1e1c61bd7c7a50cad1fda62588f2f2073be36e4a73f3c83a WatchSource:0}: Error finding container 50121931fa8e646b1e1c61bd7c7a50cad1fda62588f2f2073be36e4a73f3c83a: Status 404 returned error can't find the container with id 50121931fa8e646b1e1c61bd7c7a50cad1fda62588f2f2073be36e4a73f3c83a
Sep 18 20:13:25.177660 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:25.177617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl" event={"ID":"488db7c3-8e3a-424e-bc1e-6c98c08a9e88","Type":"ContainerStarted","Data":"37c38c8ef4c062978b3cb72803c1a52a0e2a44e66ffacb47ff70e83ecdca39c2"}
Sep 18 20:13:25.178228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:25.177688    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl" event={"ID":"488db7c3-8e3a-424e-bc1e-6c98c08a9e88","Type":"ContainerStarted","Data":"50121931fa8e646b1e1c61bd7c7a50cad1fda62588f2f2073be36e4a73f3c83a"}
Sep 18 20:13:25.196022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:25.195962    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl" podStartSLOduration=2.034480598 podStartE2EDuration="2.195940261s" totalImagesPullingTime="161.459663ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:13:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:13:25.192950167 +0000 UTC m=+749.322252118" watchObservedRunningTime="2026-09-18 20:13:25.195940261 +0000 UTC m=+749.325242214"
Sep 18 20:13:27.818757 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:27.818712    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:27.819312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:27.818784    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:27.831138 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:27.831115    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:28.202985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:28.202965    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-87gsc"
Sep 18 20:13:34.308291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:34.308261    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:34.308692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:34.308317    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:34.320842 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:34.320819    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:13:35.237512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:13:35.237482    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-84bjl"
Sep 18 20:14:00.790780 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.790746    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"]
Sep 18 20:14:00.797817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.797794    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.800422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.800397    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\""
Sep 18 20:14:00.805968 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.805938    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"]
Sep 18 20:14:00.854989 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.854957    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.855205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.855029    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.855205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.855130    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.855205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.855169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.855376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.855201    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbmn4\" (UniqueName: \"kubernetes.io/projected/b4d7ef67-f317-49c9-839c-4d455bfd37a7-kube-api-access-lbmn4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.855376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.855247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956438    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956703 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956509    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956703 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956577    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956694    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956740    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956772    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbmn4\" (UniqueName: \"kubernetes.io/projected/b4d7ef67-f317-49c9-839c-4d455bfd37a7-kube-api-access-lbmn4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.956988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.956970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.957124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.957099    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.957211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.957158    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.958866 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.958843    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b4d7ef67-f317-49c9-839c-4d455bfd37a7-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.959226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.959205    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b4d7ef67-f317-49c9-839c-4d455bfd37a7-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:00.966191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:00.966165    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbmn4\" (UniqueName: \"kubernetes.io/projected/b4d7ef67-f317-49c9-839c-4d455bfd37a7-kube-api-access-lbmn4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-97kjp\" (UID: \"b4d7ef67-f317-49c9-839c-4d455bfd37a7\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:01.112291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:01.112203    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:01.385595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:01.385569    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"]
Sep 18 20:14:01.387222 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:14:01.387189    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4d7ef67_f317_49c9_839c_4d455bfd37a7.slice/crio-d3721593aba7000b3e53f2a2d137a3a8c703ea91f1f6b19954514959f4d589bc WatchSource:0}: Error finding container d3721593aba7000b3e53f2a2d137a3a8c703ea91f1f6b19954514959f4d589bc: Status 404 returned error can't find the container with id d3721593aba7000b3e53f2a2d137a3a8c703ea91f1f6b19954514959f4d589bc
Sep 18 20:14:02.368911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:02.368876    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp" event={"ID":"b4d7ef67-f317-49c9-839c-4d455bfd37a7","Type":"ContainerStarted","Data":"b3e62f2119e7d79426a6f68596c3df99d2c4f56163b96d0d0c3a658d4ad3e77c"}
Sep 18 20:14:02.369317 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:02.368931    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp" event={"ID":"b4d7ef67-f317-49c9-839c-4d455bfd37a7","Type":"ContainerStarted","Data":"d3721593aba7000b3e53f2a2d137a3a8c703ea91f1f6b19954514959f4d589bc"}
Sep 18 20:14:02.384630 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:02.384572    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp" podStartSLOduration=2.199887352 podStartE2EDuration="2.384551192s" totalImagesPullingTime="184.66384ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:14:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:14:02.383906351 +0000 UTC m=+786.513208303" watchObservedRunningTime="2026-09-18 20:14:02.384551192 +0000 UTC m=+786.513853143"
Sep 18 20:14:11.112807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:11.112723    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:11.112807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:11.112779    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:11.125128 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:11.125106    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:14:11.430300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:14:11.430219    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-97kjp"
Sep 18 20:15:00.143456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.143424    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"]
Sep 18 20:15:00.147039 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.147022    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"
Sep 18 20:15:00.149330 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.149308    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:15:00.155698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.155676    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"]
Sep 18 20:15:00.158486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.158465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"
Sep 18 20:15:00.430537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.430513    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"]
Sep 18 20:15:00.432225 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:15:00.432186    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6767e001_775f_41b7_a3eb_7b587ce9a2db.slice/crio-525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d WatchSource:0}: Error finding container 525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d: Status 404 returned error can't find the container with id 525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d
Sep 18 20:15:00.670113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:00.670081    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb" event={"ID":"6767e001-775f-41b7-a3eb-7b587ce9a2db","Type":"ContainerStarted","Data":"525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d"}
Sep 18 20:15:03.687963 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:03.687936    2566 generic.go:356] "Generic (PLEG): container finished" podID="6767e001-775f-41b7-a3eb-7b587ce9a2db" containerID="d5d2d083eeba73909c29177cfcf513945dd1abbb9b0dd7ce1a2cb96ff60a163a" exitCode=0
Sep 18 20:15:03.688278 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:03.688028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb" event={"ID":"6767e001-775f-41b7-a3eb-7b587ce9a2db","Type":"ContainerDied","Data":"d5d2d083eeba73909c29177cfcf513945dd1abbb9b0dd7ce1a2cb96ff60a163a"}
Sep 18 20:15:04.821669 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:04.821645    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"
Sep 18 20:15:05.699885 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:05.699853    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb" event={"ID":"6767e001-775f-41b7-a3eb-7b587ce9a2db","Type":"ContainerDied","Data":"525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d"}
Sep 18 20:15:05.699885 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:05.699869    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829375-nfbjb"
Sep 18 20:15:05.700124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:05.699900    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="525bfeb48af2da605afd873ad86250dc4ec426a4845f828f5e63ff8e4ff1ff3d"
Sep 18 20:15:51.266966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.266933    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:15:51.267408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.267146    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-d96447cdd-kb9wl" podUID="a6027356-c780-4fef-90bd-f9d9674e7bef" containerName="authorino" containerID="cri-o://e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a" gracePeriod=30
Sep 18 20:15:51.519118 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.519069    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:15:51.628704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.628675    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert\") pod \"a6027356-c780-4fef-90bd-f9d9674e7bef\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") "
Sep 18 20:15:51.628815 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.628760    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m26zd\" (UniqueName: \"kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd\") pod \"a6027356-c780-4fef-90bd-f9d9674e7bef\" (UID: \"a6027356-c780-4fef-90bd-f9d9674e7bef\") "
Sep 18 20:15:51.630727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.630701    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd" (OuterVolumeSpecName: "kube-api-access-m26zd") pod "a6027356-c780-4fef-90bd-f9d9674e7bef" (UID: "a6027356-c780-4fef-90bd-f9d9674e7bef"). InnerVolumeSpecName "kube-api-access-m26zd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:15:51.639072 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.639031    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "a6027356-c780-4fef-90bd-f9d9674e7bef" (UID: "a6027356-c780-4fef-90bd-f9d9674e7bef"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:15:51.729598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.729573    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m26zd\" (UniqueName: \"kubernetes.io/projected/a6027356-c780-4fef-90bd-f9d9674e7bef-kube-api-access-m26zd\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:15:51.729598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.729598    2566 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/a6027356-c780-4fef-90bd-f9d9674e7bef-tls-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:15:51.929507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.929440    2566 generic.go:356] "Generic (PLEG): container finished" podID="a6027356-c780-4fef-90bd-f9d9674e7bef" containerID="e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a" exitCode=0
Sep 18 20:15:51.929507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.929482    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-d96447cdd-kb9wl"
Sep 18 20:15:51.929661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.929526    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d96447cdd-kb9wl" event={"ID":"a6027356-c780-4fef-90bd-f9d9674e7bef","Type":"ContainerDied","Data":"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"}
Sep 18 20:15:51.929661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.929572    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-d96447cdd-kb9wl" event={"ID":"a6027356-c780-4fef-90bd-f9d9674e7bef","Type":"ContainerDied","Data":"f1fb0751dfaf4b4980dc5271230ebaad30003c45b9a6912a74e8a1112e451fb0"}
Sep 18 20:15:51.929661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.929591    2566 scope.go:117] "RemoveContainer" containerID="e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"
Sep 18 20:15:51.938726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.938709    2566 scope.go:117] "RemoveContainer" containerID="e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"
Sep 18 20:15:51.938997 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:15:51.938978    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a\": container with ID starting with e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a not found: ID does not exist" containerID="e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"
Sep 18 20:15:51.939042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.939004    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a"} err="failed to get container status \"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a\": rpc error: code = NotFound desc = could not find container \"e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a\": container with ID starting with e7f6db46f46983d51bebd8bdbcd61da96a057d5beeb7fe094bfceb803bf0970a not found: ID does not exist"
Sep 18 20:15:51.950364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.950343    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:15:51.955676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:51.955658    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-d96447cdd-kb9wl"]
Sep 18 20:15:52.086399 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.086372    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6027356-c780-4fef-90bd-f9d9674e7bef" path="/var/lib/kubelet/pods/a6027356-c780-4fef-90bd-f9d9674e7bef/volumes"
Sep 18 20:15:52.141357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.141328    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:15:52.141650 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.141623    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" podUID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" containerName="payload-pre-processing" containerID="cri-o://ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e" gracePeriod=30
Sep 18 20:15:52.397237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.397215    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:15:52.536603 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.536536    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume\") pod \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") "
Sep 18 20:15:52.536733 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.536619    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2xcg4\" (UniqueName: \"kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4\") pod \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\" (UID: \"76d9426f-4053-4155-8700-cb7dc1aa4a6c\") "
Sep 18 20:15:52.536937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.536914    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "76d9426f-4053-4155-8700-cb7dc1aa4a6c" (UID: "76d9426f-4053-4155-8700-cb7dc1aa4a6c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:15:52.538609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.538577    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4" (OuterVolumeSpecName: "kube-api-access-2xcg4") pod "76d9426f-4053-4155-8700-cb7dc1aa4a6c" (UID: "76d9426f-4053-4155-8700-cb7dc1aa4a6c"). InnerVolumeSpecName "kube-api-access-2xcg4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:15:52.638013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.637978    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2xcg4\" (UniqueName: \"kubernetes.io/projected/76d9426f-4053-4155-8700-cb7dc1aa4a6c-kube-api-access-2xcg4\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:15:52.638013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.638011    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/76d9426f-4053-4155-8700-cb7dc1aa4a6c-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:15:52.936190 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.936120    2566 generic.go:356] "Generic (PLEG): container finished" podID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" containerID="ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e" exitCode=0
Sep 18 20:15:52.936190 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.936169    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"
Sep 18 20:15:52.936383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.936210    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" event={"ID":"76d9426f-4053-4155-8700-cb7dc1aa4a6c","Type":"ContainerDied","Data":"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"}
Sep 18 20:15:52.936383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.936267    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v" event={"ID":"76d9426f-4053-4155-8700-cb7dc1aa4a6c","Type":"ContainerDied","Data":"3315d8ec6592a511a0c18588d2e3018787bbfa049ba97a7a231feeabd53f19ee"}
Sep 18 20:15:52.936383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.936295    2566 scope.go:117] "RemoveContainer" containerID="ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"
Sep 18 20:15:52.945927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.945907    2566 scope.go:117] "RemoveContainer" containerID="ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"
Sep 18 20:15:52.946251 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:15:52.946232    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e\": container with ID starting with ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e not found: ID does not exist" containerID="ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"
Sep 18 20:15:52.946322 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.946258    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e"} err="failed to get container status \"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e\": rpc error: code = NotFound desc = could not find container \"ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e\": container with ID starting with ba3ccc0b66ce3c475a9303a1d844c10ad33143af3836a1e8c932a3c1af5a9c5e not found: ID does not exist"
Sep 18 20:15:52.958117 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.958088    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:15:52.960689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:52.960669    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-59fbc47bf8-bhm7v"]
Sep 18 20:15:54.080692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:54.080657    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" path="/var/lib/kubelet/pods/76d9426f-4053-4155-8700-cb7dc1aa4a6c/volumes"
Sep 18 20:15:58.068642 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:58.068610    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:15:58.069090 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:15:58.068696    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:16:13.152976 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.152895    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"]
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153283    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6767e001-775f-41b7-a3eb-7b587ce9a2db" containerName="cleanup"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153294    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6767e001-775f-41b7-a3eb-7b587ce9a2db" containerName="cleanup"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153307    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" containerName="payload-pre-processing"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153313    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" containerName="payload-pre-processing"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153322    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6027356-c780-4fef-90bd-f9d9674e7bef" containerName="authorino"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153328    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6027356-c780-4fef-90bd-f9d9674e7bef" containerName="authorino"
Sep 18 20:16:13.153387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153384    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6767e001-775f-41b7-a3eb-7b587ce9a2db" containerName="cleanup"
Sep 18 20:16:13.153593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153392    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="76d9426f-4053-4155-8700-cb7dc1aa4a6c" containerName="payload-pre-processing"
Sep 18 20:16:13.153593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.153399    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6027356-c780-4fef-90bd-f9d9674e7bef" containerName="authorino"
Sep 18 20:16:13.156204 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.156188    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.158276 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.158257    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 18 20:16:13.166826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.166805    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"]
Sep 18 20:16:13.310144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.310093    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwzzc\" (UniqueName: \"kubernetes.io/projected/896a9164-73dd-44bb-81ea-e8ea10f76ce6-kube-api-access-mwzzc\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.310323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.310258    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896a9164-73dd-44bb-81ea-e8ea10f76ce6-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.411419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.411333    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwzzc\" (UniqueName: \"kubernetes.io/projected/896a9164-73dd-44bb-81ea-e8ea10f76ce6-kube-api-access-mwzzc\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.411419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.411385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896a9164-73dd-44bb-81ea-e8ea10f76ce6-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.412194 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.412176    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/896a9164-73dd-44bb-81ea-e8ea10f76ce6-plugins-config-volume\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.419160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.419143    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwzzc\" (UniqueName: \"kubernetes.io/projected/896a9164-73dd-44bb-81ea-e8ea10f76ce6-kube-api-access-mwzzc\") pod \"payload-pre-processing-76f657cdc4-rwgf8\" (UID: \"896a9164-73dd-44bb-81ea-e8ea10f76ce6\") " pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.467927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.467903    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:13.736406 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:13.736385    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"]
Sep 18 20:16:13.738441 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:13.738412    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod896a9164_73dd_44bb_81ea_e8ea10f76ce6.slice/crio-82a0fa2a07b73c1eacd218624f5b119ffd186d8041f309812d7f19d55c72dea1 WatchSource:0}: Error finding container 82a0fa2a07b73c1eacd218624f5b119ffd186d8041f309812d7f19d55c72dea1: Status 404 returned error can't find the container with id 82a0fa2a07b73c1eacd218624f5b119ffd186d8041f309812d7f19d55c72dea1
Sep 18 20:16:14.048072 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:14.047973    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8" event={"ID":"896a9164-73dd-44bb-81ea-e8ea10f76ce6","Type":"ContainerStarted","Data":"82a0fa2a07b73c1eacd218624f5b119ffd186d8041f309812d7f19d55c72dea1"}
Sep 18 20:16:16.060031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:16.059946    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8" event={"ID":"896a9164-73dd-44bb-81ea-e8ea10f76ce6","Type":"ContainerStarted","Data":"8f3212830e7fc3c86c036d7200a85ba5f2dbe1f54c7361dcf289fab98a66b0d1"}
Sep 18 20:16:16.060520 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:16.060097    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:16.076144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:16.076087    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8" podStartSLOduration=1.00552602 podStartE2EDuration="3.076072913s" totalImagesPullingTime="2.070546893s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:13 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:16.075636099 +0000 UTC m=+920.204938050" watchObservedRunningTime="2026-09-18 20:16:16.076072913 +0000 UTC m=+920.205374854"
Sep 18 20:16:27.066724 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:27.066690    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-76f657cdc4-rwgf8"
Sep 18 20:16:50.597706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.597670    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:16:50.606238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.606214    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.608547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.608527    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-inf-a-e2dea3-openshift-default-dockercfg-vhpqc\""
Sep 18 20:16:50.612535 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.612515    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:16:50.661279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.661247    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:16:50.666102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.666068    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.668439 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.668418    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w3-b88002-openshift-default-dockercfg-ntx86\""
Sep 18 20:16:50.676618 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.676567    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:16:50.718503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718479    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dt76\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718511    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718527    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718545    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718584    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718667    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718692    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718775    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9tbks\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718797    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718818    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718838    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718860    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.718886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718883    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.719219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.719219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718933    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.719219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.718994    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.719219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.719067    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.795427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.795395    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:16:50.806572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.806549    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.809694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.809670    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-116778-openshift-default-dockercfg-2dkmn\""
Sep 18 20:16:50.820089 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9tbks\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820107    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820139    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820173    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820204    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820228    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820280    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820475 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820323    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820475 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820395    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820475 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820440    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820497    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5dt76\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820553    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820586    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820608    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820619    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820622    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820641    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820704    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820825    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.820988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.820869    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.821396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.821035    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.821396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.821263    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.821396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.821375    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.821570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.821508    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.821677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.821658    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.822590 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.822565    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.822886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.822868    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.823082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.823049    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.823366 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.823350    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.832845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.832826    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9tbks\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.841437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.841115    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:16:50.848665 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.848612    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token\") pod \"e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") " pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.848742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.848712    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.857525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.857499    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dt76\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76\") pod \"e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") " pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:50.919074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.919029    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:50.921440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921421    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-89ghg\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921452    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921472    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921511    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921606 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921594    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921617    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921675 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921658    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921708 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.921741 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.921718    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:50.977518 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:50.977468    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:51.022985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.022950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-89ghg\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.022995    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023094    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023172    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023256    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023293    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023328    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023743 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023719    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023860    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.023948 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.023891    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.024047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.024027    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.024268 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.024251    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.025454 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.025431    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.026172 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.026126    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.031964 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.031486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-89ghg\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.034232 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.032647    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token\") pod \"e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") " pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.117104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.117023    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:51.197610 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.197576    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:16:51.199150 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:51.199122    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24891591_0b52_493b_8eb6_f0db69e999f0.slice/crio-8f1df4953d36eb1b72d67c098a253396159cfd9dd90305668ce0eeb66074a0fd WatchSource:0}: Error finding container 8f1df4953d36eb1b72d67c098a253396159cfd9dd90305668ce0eeb66074a0fd: Status 404 returned error can't find the container with id 8f1df4953d36eb1b72d67c098a253396159cfd9dd90305668ce0eeb66074a0fd
Sep 18 20:16:51.201339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.201311    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.201418 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.201369    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.201418 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.201398    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.225633 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.225612    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerStarted","Data":"8f1df4953d36eb1b72d67c098a253396159cfd9dd90305668ce0eeb66074a0fd"}
Sep 18 20:16:51.257602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.257583    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:16:51.259437 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:51.259409    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd640ffa4_03df_4e5a_86c7_582d2682a1f9.slice/crio-f8d8bd9d3bbd4cd531370b25feca7b2be46d6dc43c30bca4eeb94dd4804c104f WatchSource:0}: Error finding container f8d8bd9d3bbd4cd531370b25feca7b2be46d6dc43c30bca4eeb94dd4804c104f: Status 404 returned error can't find the container with id f8d8bd9d3bbd4cd531370b25feca7b2be46d6dc43c30bca4eeb94dd4804c104f
Sep 18 20:16:51.261667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.261638    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.261738 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.261722    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.261818 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.261771    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.397239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.397208    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:16:51.399444 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:51.399417    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod432fe170_b3b7_45a8_a573_5d45a98a8f96.slice/crio-5a2feae6f852483b5548a3044eeb5caaa7aa401c5ce5dfdbb05d06a52f057bdf WatchSource:0}: Error finding container 5a2feae6f852483b5548a3044eeb5caaa7aa401c5ce5dfdbb05d06a52f057bdf: Status 404 returned error can't find the container with id 5a2feae6f852483b5548a3044eeb5caaa7aa401c5ce5dfdbb05d06a52f057bdf
Sep 18 20:16:51.402858 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.402777    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.402944 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.402862    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:51.402944 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:51.402907    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:16:52.232557 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.232521    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerStarted","Data":"320e5e681308ab19f7160c44ab3921e98d68ccbf3dd3c45a634aead7b1ba97a8"}
Sep 18 20:16:52.232996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.232587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerStarted","Data":"f8d8bd9d3bbd4cd531370b25feca7b2be46d6dc43c30bca4eeb94dd4804c104f"}
Sep 18 20:16:52.234323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.234290    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerStarted","Data":"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"}
Sep 18 20:16:52.236038 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.236015    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerStarted","Data":"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"}
Sep 18 20:16:52.236152 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.236080    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerStarted","Data":"5a2feae6f852483b5548a3044eeb5caaa7aa401c5ce5dfdbb05d06a52f057bdf"}
Sep 18 20:16:52.256596 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.254529    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podStartSLOduration=2.2545090979999998 podStartE2EDuration="2.254509098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:52.250999138 +0000 UTC m=+956.380301090" watchObservedRunningTime="2026-09-18 20:16:52.254509098 +0000 UTC m=+956.383811049"
Sep 18 20:16:52.277400 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.277361    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podStartSLOduration=2.277347559 podStartE2EDuration="2.277347559s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:52.274191141 +0000 UTC m=+956.403493092" watchObservedRunningTime="2026-09-18 20:16:52.277347559 +0000 UTC m=+956.406649509"
Sep 18 20:16:52.291960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.291923    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podStartSLOduration=2.291909978 podStartE2EDuration="2.291909978s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:52.290712638 +0000 UTC m=+956.420014599" watchObservedRunningTime="2026-09-18 20:16:52.291909978 +0000 UTC m=+956.421211928"
Sep 18 20:16:52.919490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.919459    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:52.924258 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.924231    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:52.978488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.978466    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:52.983401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:52.983381    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:53.117953 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.117927    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:53.122661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.122641    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:53.242204 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.242177    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:53.242624 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.242519    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:53.242624 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.242568    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:53.242919 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.242902    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:16:53.243122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.243103    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:16:53.243191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:53.243152    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:16:57.785343 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.785309    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:16:57.789199 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.789178    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:57.792761 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.792744    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e2dea3-dockercfg-smdkp\""
Sep 18 20:16:57.792761 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.792757    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e2dea3\""
Sep 18 20:16:57.808602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.808577    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:16:57.838550 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.838526    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:16:57.842436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.842414    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:57.865493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.865466    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:16:57.893484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.893450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:57.893744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.893714    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b2g64\" (UniqueName: \"kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:57.994379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.994347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkccp\" (UniqueName: \"kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:57.994379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.994385    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b2g64\" (UniqueName: \"kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:57.994622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.994411    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:57.994622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.994510    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:57.996578 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:57.996563    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-a-e2dea3\""
Sep 18 20:16:58.002413 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.002398    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b2g64\" (UniqueName: \"kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:58.005065 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.005033    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:58.095373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.095277    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:58.095533 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.095420    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkccp\" (UniqueName: \"kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:58.096310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.096278    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:58.101981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.101957    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-a-e2dea3-dockercfg-smdkp\""
Sep 18 20:16:58.108705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.108685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkccp\" (UniqueName: \"kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp\") pod \"payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") " pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:58.110377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.110362    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:58.153228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.153199    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:58.387660 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.387636    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:16:58.389593 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:58.389568    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod467b902a_a909_4e80_90e8_fc211b517ced.slice/crio-94de9d53bf512d237f2f36c30f804c9c861dfbaf10261ea7e82a70045e405fc8 WatchSource:0}: Error finding container 94de9d53bf512d237f2f36c30f804c9c861dfbaf10261ea7e82a70045e405fc8: Status 404 returned error can't find the container with id 94de9d53bf512d237f2f36c30f804c9c861dfbaf10261ea7e82a70045e405fc8
Sep 18 20:16:58.464163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.464138    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:16:58.466176 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:58.466150    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8a7e6ff_7d07_45a8_b875_11bfb5252d74.slice/crio-426b1681ff124bb8e7a2d9c8e6a40ed60f5aad0a96c43d867f5adeb6fb69aaa9 WatchSource:0}: Error finding container 426b1681ff124bb8e7a2d9c8e6a40ed60f5aad0a96c43d867f5adeb6fb69aaa9: Status 404 returned error can't find the container with id 426b1681ff124bb8e7a2d9c8e6a40ed60f5aad0a96c43d867f5adeb6fb69aaa9
Sep 18 20:16:58.776800 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.776770    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:16:58.780475 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.780455    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:58.782904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.782883    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-worker-w0-116778\""
Sep 18 20:16:58.789732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.789708    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:16:58.903041 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.903002    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:58.903355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:58.903330    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5grhg\" (UniqueName: \"kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.004279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.004242    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.004473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.004289    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5grhg\" (UniqueName: \"kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.005131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.005105    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.012799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.012771    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5grhg\" (UniqueName: \"kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg\") pod \"payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") " pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.092486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.092396    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:16:59.288657 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.288620    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" event={"ID":"467b902a-a909-4e80-90e8-fc211b517ced","Type":"ContainerStarted","Data":"5713188acf52d50ca7b382efee4b252a5345556aa58165794bb35179d1380f44"}
Sep 18 20:16:59.288848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.288711    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" event={"ID":"467b902a-a909-4e80-90e8-fc211b517ced","Type":"ContainerStarted","Data":"94de9d53bf512d237f2f36c30f804c9c861dfbaf10261ea7e82a70045e405fc8"}
Sep 18 20:16:59.288848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.288739    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:16:59.290216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.290188    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" event={"ID":"e8a7e6ff-7d07-45a8-b875-11bfb5252d74","Type":"ContainerStarted","Data":"f52f260e7a5ab1a46c468d45807a5188efa3e97176de34972085d73abbee993e"}
Sep 18 20:16:59.290324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.290229    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" event={"ID":"e8a7e6ff-7d07-45a8-b875-11bfb5252d74","Type":"ContainerStarted","Data":"426b1681ff124bb8e7a2d9c8e6a40ed60f5aad0a96c43d867f5adeb6fb69aaa9"}
Sep 18 20:16:59.290324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.290304    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:16:59.315928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.315876    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" podStartSLOduration=2.315855358 podStartE2EDuration="2.315855358s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:59.315711011 +0000 UTC m=+963.445012963" watchObservedRunningTime="2026-09-18 20:16:59.315855358 +0000 UTC m=+963.445157309"
Sep 18 20:16:59.367528 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.367127    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" podStartSLOduration=2.367110053 podStartE2EDuration="2.367110053s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:16:59.34392276 +0000 UTC m=+963.473224715" watchObservedRunningTime="2026-09-18 20:16:59.367110053 +0000 UTC m=+963.496412003"
Sep 18 20:16:59.368546 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:16:59.368528    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:16:59.370517 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:16:59.370486    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b2f25b8_4b44_4a9e_bbf7_a4f20996d675.slice/crio-d912297c8f2063c563b798ae74f2cd9775851c11c4808a87097a38556be58c60 WatchSource:0}: Error finding container d912297c8f2063c563b798ae74f2cd9775851c11c4808a87097a38556be58c60: Status 404 returned error can't find the container with id d912297c8f2063c563b798ae74f2cd9775851c11c4808a87097a38556be58c60
Sep 18 20:17:00.297396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:00.297355    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" event={"ID":"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675","Type":"ContainerStarted","Data":"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"}
Sep 18 20:17:00.297860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:00.297416    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" event={"ID":"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675","Type":"ContainerStarted","Data":"d912297c8f2063c563b798ae74f2cd9775851c11c4808a87097a38556be58c60"}
Sep 18 20:17:00.297860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:00.297761    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:17:00.319512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:00.319462    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" podStartSLOduration=2.319444975 podStartE2EDuration="2.319444975s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:16:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:00.317201235 +0000 UTC m=+964.446503208" watchObservedRunningTime="2026-09-18 20:17:00.319444975 +0000 UTC m=+964.448746927"
Sep 18 20:17:08.445779 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.445723    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:08.450127 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.450102    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.453015 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.452991    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-7626fb93-dockercfg-kz7pl\""
Sep 18 20:17:08.453407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.453380    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-7626fb93\""
Sep 18 20:17:08.458886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.458865    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:08.476113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.476082    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:08.480479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.480458    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.490248 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.490227    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:08.522960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.522924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pt7j7\" (UniqueName: \"kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.523124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.522977    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.523187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.523129    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q7q5\" (UniqueName: \"kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.523235 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.523185    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.624580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.624540    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.624750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.624615    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pt7j7\" (UniqueName: \"kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.624750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.624644    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.624750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.624736    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4q7q5\" (UniqueName: \"kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.625413 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.625389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.625523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.625471    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.633103 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.633079    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pt7j7\" (UniqueName: \"kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7\") pod \"payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.633239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.633220    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q7q5\" (UniqueName: \"kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5\") pod \"payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") " pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:08.764831 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.764787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:08.794232 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:08.793767    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:09.051019 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.050992    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:09.052247 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:09.052220    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0e7d705_7f3a_4259_8a5e_68c3d5f02481.slice/crio-a5c561ecae314240ce1c788204333651b065c6cf36c5b59fc37929600bab0a35 WatchSource:0}: Error finding container a5c561ecae314240ce1c788204333651b065c6cf36c5b59fc37929600bab0a35: Status 404 returned error can't find the container with id a5c561ecae314240ce1c788204333651b065c6cf36c5b59fc37929600bab0a35
Sep 18 20:17:09.088298 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.088269    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:09.090196 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:09.090172    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26a81e3b_9696_4cdb_941a_b125bd74c644.slice/crio-bedfe7be390f1775620385c8958a0f117f62724b04db58bd9cb2c0dc9cfb52ec WatchSource:0}: Error finding container bedfe7be390f1775620385c8958a0f117f62724b04db58bd9cb2c0dc9cfb52ec: Status 404 returned error can't find the container with id bedfe7be390f1775620385c8958a0f117f62724b04db58bd9cb2c0dc9cfb52ec
Sep 18 20:17:09.283275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.283233    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 20:17:09.283452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.283291    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:17:09.283452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.283235    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 20:17:09.283452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.283388    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:17:09.352759 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.352663    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" event={"ID":"e0e7d705-7f3a-4259-8a5e-68c3d5f02481","Type":"ContainerStarted","Data":"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"}
Sep 18 20:17:09.352759 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.352745    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" event={"ID":"e0e7d705-7f3a-4259-8a5e-68c3d5f02481","Type":"ContainerStarted","Data":"a5c561ecae314240ce1c788204333651b065c6cf36c5b59fc37929600bab0a35"}
Sep 18 20:17:09.353108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.353043    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:09.355093 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.355040    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" event={"ID":"26a81e3b-9696-4cdb-941a-b125bd74c644","Type":"ContainerStarted","Data":"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"}
Sep 18 20:17:09.355218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.355095    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" event={"ID":"26a81e3b-9696-4cdb-941a-b125bd74c644","Type":"ContainerStarted","Data":"bedfe7be390f1775620385c8958a0f117f62724b04db58bd9cb2c0dc9cfb52ec"}
Sep 18 20:17:09.355493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.355434    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:09.383763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.383686    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" podStartSLOduration=1.383665076 podStartE2EDuration="1.383665076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:09.379493174 +0000 UTC m=+973.508795125" watchObservedRunningTime="2026-09-18 20:17:09.383665076 +0000 UTC m=+973.512967029"
Sep 18 20:17:09.408108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:09.408028    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" podStartSLOduration=1.40801282 podStartE2EDuration="1.40801282s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:09.4064794 +0000 UTC m=+973.535781352" watchObservedRunningTime="2026-09-18 20:17:09.40801282 +0000 UTC m=+973.537314809"
Sep 18 20:17:10.299767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.299735    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:17:10.300189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.300170    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:17:10.372860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.372827    2566 generic.go:356] "Generic (PLEG): container finished" podID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerID="320e5e681308ab19f7160c44ab3921e98d68ccbf3dd3c45a634aead7b1ba97a8" exitCode=0
Sep 18 20:17:10.373077 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.372903    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerDied","Data":"320e5e681308ab19f7160c44ab3921e98d68ccbf3dd3c45a634aead7b1ba97a8"}
Sep 18 20:17:10.373368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.373352    2566 scope.go:117] "RemoveContainer" containerID="320e5e681308ab19f7160c44ab3921e98d68ccbf3dd3c45a634aead7b1ba97a8"
Sep 18 20:17:10.374696 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.374669    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:10.374785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.374751    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:10.374785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.374760    2566 generic.go:356] "Generic (PLEG): container finished" podID="24891591-0b52-493b-8eb6-f0db69e999f0" containerID="bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616" exitCode=0
Sep 18 20:17:10.374911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.374789    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:10.374911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.374832    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerDied","Data":"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"}
Sep 18 20:17:10.375448 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.375427    2566 scope.go:117] "RemoveContainer" containerID="bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"
Sep 18 20:17:10.376541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.376513    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:10.376632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.376575    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:10.376632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:10.376608    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:11.304210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.304178    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:17:11.384429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.384390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerStarted","Data":"ed4bebbe202ea21a0484185e3ad2cc8658766c782e40813cb0d6adc56f1fccfe"}
Sep 18 20:17:11.386785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.386761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerStarted","Data":"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"}
Sep 18 20:17:11.920404 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.920358    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:17:11.920625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.920509    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:11.920625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.920574    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:11.978390 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.978345    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:17:11.978820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.978786    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:11.979008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:11.978846    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:12.919474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:12.919428    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:12.919838 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:12.919500    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:12.978162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:12.978126    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:12.978434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:12.978405    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:13.920396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:13.920360    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:13.920764 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:13.920412    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:13.978080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:13.977900    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:13.978080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:13.977985    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:14.920860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:14.920399    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:14.920860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:14.920475    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:14.978775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:14.978344    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:14.978775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:14.978406    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:15.919602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:15.919563    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:15.919776 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:15.919631    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:15.978529 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:15.978482    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:15.979019 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:15.978556    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:16.664162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.664114    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:17:16.670898 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.670872    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.676145 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.676124    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-a85f8b\""
Sep 18 20:17:16.676552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.676534    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-inf-b-a85f8b-dockercfg-bfpfq\""
Sep 18 20:17:16.683848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.683448    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:17:16.696139 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.687088    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:17:16.700368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.699795    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.714191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.714166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:17:16.772022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.771625    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.772022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.771758    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.772022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.771813    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thxwd\" (UniqueName: \"kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.772022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.771854    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9cnq\" (UniqueName: \"kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.872744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.872708    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.873025 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.873001    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thxwd\" (UniqueName: \"kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.873208 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.873191    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9cnq\" (UniqueName: \"kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.873397 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.873381    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.874422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.874391    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.875338 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.875317    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.889308 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.889288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thxwd\" (UniqueName: \"kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd\") pod \"payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") " pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:16.890119 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.890042    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9cnq\" (UniqueName: \"kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq\") pod \"payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:16.920155 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.920043    2566 patch_prober.go:28] interesting pod/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused" start-of-body=
Sep 18 20:17:16.920155 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.920140    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.46:15021/healthz/ready\": dial tcp 10.132.0.46:15021: connect: connection refused"
Sep 18 20:17:16.978160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.978126    2566 patch_prober.go:28] interesting pod/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused" start-of-body=
Sep 18 20:17:16.978351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.978187    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.47:15021/healthz/ready\": dial tcp 10.132.0.47:15021: connect: connection refused"
Sep 18 20:17:16.997197 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:16.997162    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:17.021608 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.021575    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:17.322169 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.322135    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:17:17.325518 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:17.325493    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb0f727_3591_4cfd_8f20_7eff6df56851.slice/crio-11d9092c0e5e3f305cde0a1179b186f580aedc4326d6d65d63db475e972b17b6 WatchSource:0}: Error finding container 11d9092c0e5e3f305cde0a1179b186f580aedc4326d6d65d63db475e972b17b6: Status 404 returned error can't find the container with id 11d9092c0e5e3f305cde0a1179b186f580aedc4326d6d65d63db475e972b17b6
Sep 18 20:17:17.352481 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.352338    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:17:17.354770 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:17.354578    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod587cadfb_dd4d_4610_b2d4_93316abd4d34.slice/crio-d2119f6f300d99e9b0197741eeecf280257e89842c55ea417df2c1ccd60d356d WatchSource:0}: Error finding container d2119f6f300d99e9b0197741eeecf280257e89842c55ea417df2c1ccd60d356d: Status 404 returned error can't find the container with id d2119f6f300d99e9b0197741eeecf280257e89842c55ea417df2c1ccd60d356d
Sep 18 20:17:17.456747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.456709    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" event={"ID":"1bb0f727-3591-4cfd-8f20-7eff6df56851","Type":"ContainerStarted","Data":"91064f9b7703dfc98bec945daea6aa4c980dbac6871e913cd665b67d96ba1301"}
Sep 18 20:17:17.456933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.456776    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" event={"ID":"1bb0f727-3591-4cfd-8f20-7eff6df56851","Type":"ContainerStarted","Data":"11d9092c0e5e3f305cde0a1179b186f580aedc4326d6d65d63db475e972b17b6"}
Sep 18 20:17:17.456933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.456821    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:17.460272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.459115    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" event={"ID":"587cadfb-dd4d-4610-b2d4-93316abd4d34","Type":"ContainerStarted","Data":"17f6d52267dec896ce138ef63b9eba50466d4b90567d629a2eecf2ec46ad5e0e"}
Sep 18 20:17:17.460272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.459167    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" event={"ID":"587cadfb-dd4d-4610-b2d4-93316abd4d34","Type":"ContainerStarted","Data":"d2119f6f300d99e9b0197741eeecf280257e89842c55ea417df2c1ccd60d356d"}
Sep 18 20:17:17.460272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.459713    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:17.484833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.484770    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" podStartSLOduration=1.4847497889999999 podStartE2EDuration="1.484749789s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:17.482048692 +0000 UTC m=+981.611350656" watchObservedRunningTime="2026-09-18 20:17:17.484749789 +0000 UTC m=+981.614051740"
Sep 18 20:17:17.504225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.503365    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" podStartSLOduration=1.503343397 podStartE2EDuration="1.503343397s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:17.502759292 +0000 UTC m=+981.632061241" watchObservedRunningTime="2026-09-18 20:17:17.503343397 +0000 UTC m=+981.632645348"
Sep 18 20:17:17.924067 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.924018    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:17:17.924363 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.924329    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:17:17.924983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.924966    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:17:17.984519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.984492    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:17:17.985278 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.985256    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:17:17.990927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:17.987778    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:17:20.377241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.377203    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:20.377663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.377421    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:20.479468 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.479441    2566 generic.go:356] "Generic (PLEG): container finished" podID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerID="fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c" exitCode=0
Sep 18 20:17:20.479670 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.479500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerDied","Data":"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"}
Sep 18 20:17:20.479894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.479880    2566 scope.go:117] "RemoveContainer" containerID="fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"
Sep 18 20:17:20.481165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.481140    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:20.481232 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.481222    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:20.481267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:20.481253    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:17:21.486922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:21.486885    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerStarted","Data":"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"}
Sep 18 20:17:22.117881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:22.117845    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:17:22.119035 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:22.119011    2566 patch_prober.go:28] interesting pod/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 18 20:17:22.119103 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:22.119069    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 18 20:17:23.118180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:23.118140    2566 patch_prober.go:28] interesting pod/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 18 20:17:23.118548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:23.118204    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 18 20:17:23.241750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:23.241710    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:17:24.117711 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:24.117677    2566 patch_prober.go:28] interesting pod/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 18 20:17:24.117877 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:24.117731    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 18 20:17:25.117952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:25.117918    2566 patch_prober.go:28] interesting pod/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 18 20:17:25.118340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:25.117979    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 18 20:17:26.118142 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:26.118108    2566 patch_prober.go:28] interesting pod/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused" start-of-body=
Sep 18 20:17:26.118503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:26.118159    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.48:15021/healthz/ready\": dial tcp 10.132.0.48:15021: connect: connection refused"
Sep 18 20:17:27.121805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:27.121775    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:17:27.122889 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:27.122868    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:17:28.467074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:28.467021    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:17:29.475648 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:29.475617    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:17:30.432484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.432449    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"]
Sep 18 20:17:30.435697 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.435677    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:30.438019 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.437997    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:17:30.438164 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.438017    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:17:30.444736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.444701    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"]
Sep 18 20:17:30.523910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.523880    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx\" (UID: \"1f05d491-11ea-4cee-9986-ab9428980898\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:30.624520 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.624490    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx\" (UID: \"1f05d491-11ea-4cee-9986-ab9428980898\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:30.625067 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.625037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx\" (UID: \"1f05d491-11ea-4cee-9986-ab9428980898\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:30.753851 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:30.753822    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:31.024469 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:31.024389    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"]
Sep 18 20:17:31.027485 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:31.027460    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f05d491_11ea_4cee_9986_ab9428980898.slice/crio-d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96 WatchSource:0}: Error finding container d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96: Status 404 returned error can't find the container with id d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96
Sep 18 20:17:31.549162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:31.549124    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx" event={"ID":"1f05d491-11ea-4cee-9986-ab9428980898","Type":"ContainerStarted","Data":"31b86a638bb3b2e533568923f9008679528fc58d78ebf756527e8b69afd6dd26"}
Sep 18 20:17:31.549592 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:31.549179    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx" event={"ID":"1f05d491-11ea-4cee-9986-ab9428980898","Type":"ContainerStarted","Data":"d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96"}
Sep 18 20:17:31.564451 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:31.564397    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx" podStartSLOduration=1.564379258 podStartE2EDuration="1.564379258s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:31.562142482 +0000 UTC m=+995.691444432" watchObservedRunningTime="2026-09-18 20:17:31.564379258 +0000 UTC m=+995.693681209"
Sep 18 20:17:32.556555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:32.556518    2566 generic.go:356] "Generic (PLEG): container finished" podID="1f05d491-11ea-4cee-9986-ab9428980898" containerID="31b86a638bb3b2e533568923f9008679528fc58d78ebf756527e8b69afd6dd26" exitCode=0
Sep 18 20:17:32.556933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:32.556600    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx" event={"ID":"1f05d491-11ea-4cee-9986-ab9428980898","Type":"ContainerDied","Data":"31b86a638bb3b2e533568923f9008679528fc58d78ebf756527e8b69afd6dd26"}
Sep 18 20:17:33.704548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:33.704526    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:33.853379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:33.853301    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca\") pod \"1f05d491-11ea-4cee-9986-ab9428980898\" (UID: \"1f05d491-11ea-4cee-9986-ab9428980898\") "
Sep 18 20:17:33.853639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:33.853615    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1f05d491-11ea-4cee-9986-ab9428980898" (UID: "1f05d491-11ea-4cee-9986-ab9428980898"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:33.954444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:33.954413    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f05d491-11ea-4cee-9986-ab9428980898-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:17:34.569663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:34.569634    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"
Sep 18 20:17:34.569859 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:34.569633    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx" event={"ID":"1f05d491-11ea-4cee-9986-ab9428980898","Type":"ContainerDied","Data":"d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96"}
Sep 18 20:17:34.569859 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:34.569755    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d9ded3d358a268f3148cf1ce9d6f8de18e980f6cf3edf6b7c8d485566b237d96"
Sep 18 20:17:37.693003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.692964    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:37.693386 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.693225    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" podUID="26a81e3b-9696-4cdb-941a-b125bd74c644" containerName="payload-processing" containerID="cri-o://48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2" gracePeriod=30
Sep 18 20:17:37.703674 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.703649    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:37.703947 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.703890    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" podUID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" containerName="payload-pre-processing" containerID="cri-o://e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4" gracePeriod=30
Sep 18 20:17:37.882810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.882765    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"]
Sep 18 20:17:37.884627 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.884603    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-7626fb93-jpkrx"]
Sep 18 20:17:37.951616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.951559    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:37.995082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:37.995040    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:38.081823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.081783    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1f05d491-11ea-4cee-9986-ab9428980898" path="/var/lib/kubelet/pods/1f05d491-11ea-4cee-9986-ab9428980898/volumes"
Sep 18 20:17:38.096309 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096286    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pt7j7\" (UniqueName: \"kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7\") pod \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") "
Sep 18 20:17:38.096460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096321    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4q7q5\" (UniqueName: \"kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5\") pod \"26a81e3b-9696-4cdb-941a-b125bd74c644\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") "
Sep 18 20:17:38.096460 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096367    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume\") pod \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\" (UID: \"e0e7d705-7f3a-4259-8a5e-68c3d5f02481\") "
Sep 18 20:17:38.096579 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096476    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume\") pod \"26a81e3b-9696-4cdb-941a-b125bd74c644\" (UID: \"26a81e3b-9696-4cdb-941a-b125bd74c644\") "
Sep 18 20:17:38.096811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096778    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e0e7d705-7f3a-4259-8a5e-68c3d5f02481" (UID: "e0e7d705-7f3a-4259-8a5e-68c3d5f02481"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:38.096811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.096796    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "26a81e3b-9696-4cdb-941a-b125bd74c644" (UID: "26a81e3b-9696-4cdb-941a-b125bd74c644"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:17:38.098521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.098498    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7" (OuterVolumeSpecName: "kube-api-access-pt7j7") pod "e0e7d705-7f3a-4259-8a5e-68c3d5f02481" (UID: "e0e7d705-7f3a-4259-8a5e-68c3d5f02481"). InnerVolumeSpecName "kube-api-access-pt7j7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:38.098601 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.098525    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5" (OuterVolumeSpecName: "kube-api-access-4q7q5") pod "26a81e3b-9696-4cdb-941a-b125bd74c644" (UID: "26a81e3b-9696-4cdb-941a-b125bd74c644"). InnerVolumeSpecName "kube-api-access-4q7q5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:17:38.198634 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.198603    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pt7j7\" (UniqueName: \"kubernetes.io/projected/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-kube-api-access-pt7j7\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:17:38.198843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.198824    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4q7q5\" (UniqueName: \"kubernetes.io/projected/26a81e3b-9696-4cdb-941a-b125bd74c644-kube-api-access-4q7q5\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:17:38.198843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.198842    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e0e7d705-7f3a-4259-8a5e-68c3d5f02481-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:17:38.198987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.198850    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/26a81e3b-9696-4cdb-941a-b125bd74c644-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:17:38.593234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.593195    2566 generic.go:356] "Generic (PLEG): container finished" podID="26a81e3b-9696-4cdb-941a-b125bd74c644" containerID="48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2" exitCode=0
Sep 18 20:17:38.593405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.593250    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"
Sep 18 20:17:38.593405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.593281    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" event={"ID":"26a81e3b-9696-4cdb-941a-b125bd74c644","Type":"ContainerDied","Data":"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"}
Sep 18 20:17:38.593405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.593371    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb" event={"ID":"26a81e3b-9696-4cdb-941a-b125bd74c644","Type":"ContainerDied","Data":"bedfe7be390f1775620385c8958a0f117f62724b04db58bd9cb2c0dc9cfb52ec"}
Sep 18 20:17:38.593405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.593400    2566 scope.go:117] "RemoveContainer" containerID="48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"
Sep 18 20:17:38.594767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.594746    2566 generic.go:356] "Generic (PLEG): container finished" podID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" containerID="e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4" exitCode=0
Sep 18 20:17:38.594886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.594805    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"
Sep 18 20:17:38.594886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.594834    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" event={"ID":"e0e7d705-7f3a-4259-8a5e-68c3d5f02481","Type":"ContainerDied","Data":"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"}
Sep 18 20:17:38.594987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.594891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb" event={"ID":"e0e7d705-7f3a-4259-8a5e-68c3d5f02481","Type":"ContainerDied","Data":"a5c561ecae314240ce1c788204333651b065c6cf36c5b59fc37929600bab0a35"}
Sep 18 20:17:38.603946 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.603929    2566 scope.go:117] "RemoveContainer" containerID="48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"
Sep 18 20:17:38.604251 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:17:38.604225    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2\": container with ID starting with 48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2 not found: ID does not exist" containerID="48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"
Sep 18 20:17:38.604370 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.604258    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2"} err="failed to get container status \"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2\": rpc error: code = NotFound desc = could not find container \"48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2\": container with ID starting with 48b3dce47be6cce5a9b8396d16c6955a8ef7377060e91abae0052fcc5e5868f2 not found: ID does not exist"
Sep 18 20:17:38.604370 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.604281    2566 scope.go:117] "RemoveContainer" containerID="e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"
Sep 18 20:17:38.612736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.612718    2566 scope.go:117] "RemoveContainer" containerID="e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"
Sep 18 20:17:38.612989 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:17:38.612969    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4\": container with ID starting with e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4 not found: ID does not exist" containerID="e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"
Sep 18 20:17:38.613067 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.612999    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4"} err="failed to get container status \"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4\": rpc error: code = NotFound desc = could not find container \"e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4\": container with ID starting with e71434cf8d7e1dcfacb6b03781b3ed0efbbf57ce9286de7038c4a158415598f4 not found: ID does not exist"
Sep 18 20:17:38.622549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.622527    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:38.628570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.628550    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-7626fb93-5c5449b869-v8nlb"]
Sep 18 20:17:38.649730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.649710    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:38.657910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:38.657889    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-7626fb93-5987c4b554-6rtnb"]
Sep 18 20:17:40.080541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:40.080511    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="26a81e3b-9696-4cdb-941a-b125bd74c644" path="/var/lib/kubelet/pods/26a81e3b-9696-4cdb-941a-b125bd74c644/volumes"
Sep 18 20:17:40.080906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:40.080874    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" path="/var/lib/kubelet/pods/e0e7d705-7f3a-4259-8a5e-68c3d5f02481/volumes"
Sep 18 20:17:43.187961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.187924    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:17:43.188609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188588    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="26a81e3b-9696-4cdb-941a-b125bd74c644" containerName="payload-processing"
Sep 18 20:17:43.188700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188611    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="26a81e3b-9696-4cdb-941a-b125bd74c644" containerName="payload-processing"
Sep 18 20:17:43.188700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188627    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1f05d491-11ea-4cee-9986-ab9428980898" containerName="revoke-keys"
Sep 18 20:17:43.188700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188634    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1f05d491-11ea-4cee-9986-ab9428980898" containerName="revoke-keys"
Sep 18 20:17:43.188700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188651    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" containerName="payload-pre-processing"
Sep 18 20:17:43.188700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188659    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" containerName="payload-pre-processing"
Sep 18 20:17:43.188939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188765    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e0e7d705-7f3a-4259-8a5e-68c3d5f02481" containerName="payload-pre-processing"
Sep 18 20:17:43.188939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188776    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1f05d491-11ea-4cee-9986-ab9428980898" containerName="revoke-keys"
Sep 18 20:17:43.188939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.188787    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="26a81e3b-9696-4cdb-941a-b125bd74c644" containerName="payload-processing"
Sep 18 20:17:43.191939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.191920    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.194403 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.194384    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-migrate-6d00b9e3\""
Sep 18 20:17:43.194537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.194427    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-migrate-6d00b9e3-dockercfg-6p66m\""
Sep 18 20:17:43.202802 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.202781    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:17:43.214310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.214289    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:17:43.217928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.217910    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.226829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.226810    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:17:43.243038 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.243000    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hsjmc\" (UniqueName: \"kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.243171 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.243083    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hkll\" (UniqueName: \"kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.243171 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.243114    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.243292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.243268    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.344143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.344096    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hsjmc\" (UniqueName: \"kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.344326 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.344168    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hkll\" (UniqueName: \"kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.344326 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.344197    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.344450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.344350    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.345326 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.345296    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.345470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.345389    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.352462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.352433    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hsjmc\" (UniqueName: \"kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc\") pod \"payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") " pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.352552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.352466    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hkll\" (UniqueName: \"kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll\") pod \"payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") " pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.504153 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.504125    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:43.528805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.528780    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:43.781765 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.781737    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:17:43.782923 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:43.782902    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod007760ab_2829_4cb9_8712_d0434e5a9d17.slice/crio-ffb14bcc7d43643739d89c74381589332195142e202c6291577d7866ed64705f WatchSource:0}: Error finding container ffb14bcc7d43643739d89c74381589332195142e202c6291577d7866ed64705f: Status 404 returned error can't find the container with id ffb14bcc7d43643739d89c74381589332195142e202c6291577d7866ed64705f
Sep 18 20:17:43.812030 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:43.812008    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:17:43.813141 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:17:43.813119    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a353a62_c591_44e1_84af_eb63da475f21.slice/crio-403a888ae03500c900066978d158887ab7b861e48135cb4873068f5e90f0d489 WatchSource:0}: Error finding container 403a888ae03500c900066978d158887ab7b861e48135cb4873068f5e90f0d489: Status 404 returned error can't find the container with id 403a888ae03500c900066978d158887ab7b861e48135cb4873068f5e90f0d489
Sep 18 20:17:44.634174 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.634134    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" event={"ID":"1a353a62-c591-44e1-84af-eb63da475f21","Type":"ContainerStarted","Data":"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"}
Sep 18 20:17:44.634620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.634205    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" event={"ID":"1a353a62-c591-44e1-84af-eb63da475f21","Type":"ContainerStarted","Data":"403a888ae03500c900066978d158887ab7b861e48135cb4873068f5e90f0d489"}
Sep 18 20:17:44.634867 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.634839    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:17:44.635926 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.635898    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" event={"ID":"007760ab-2829-4cb9-8712-d0434e5a9d17","Type":"ContainerStarted","Data":"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"}
Sep 18 20:17:44.636016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.635948    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" event={"ID":"007760ab-2829-4cb9-8712-d0434e5a9d17","Type":"ContainerStarted","Data":"ffb14bcc7d43643739d89c74381589332195142e202c6291577d7866ed64705f"}
Sep 18 20:17:44.636081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.636033    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:44.653225 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.653186    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" podStartSLOduration=1.653172571 podStartE2EDuration="1.653172571s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:44.651368606 +0000 UTC m=+1008.780670566" watchObservedRunningTime="2026-09-18 20:17:44.653172571 +0000 UTC m=+1008.782474517"
Sep 18 20:17:44.667607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:44.667569    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" podStartSLOduration=1.6675578629999999 podStartE2EDuration="1.667557863s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:17:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:17:44.666603712 +0000 UTC m=+1008.795905663" watchObservedRunningTime="2026-09-18 20:17:44.667557863 +0000 UTC m=+1008.796859812"
Sep 18 20:17:55.650098 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:55.650068    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:17:56.656723 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:17:56.656696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:18:06.286830 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.286795    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"]
Sep 18 20:18:06.290400 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.290383    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:06.292623 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.292604    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:18:06.292704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.292661    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:18:06.297361 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.297339    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"]
Sep 18 20:18:06.373825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.373795    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp\" (UID: \"21d1d885-79ec-4a07-ba11-07d49cd62697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:06.474570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.474535    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp\" (UID: \"21d1d885-79ec-4a07-ba11-07d49cd62697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:06.475195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.475171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp\" (UID: \"21d1d885-79ec-4a07-ba11-07d49cd62697\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:06.601503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.601417    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:06.876356 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:06.876328    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"]
Sep 18 20:18:06.877745 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:18:06.877714    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21d1d885_79ec_4a07_ba11_07d49cd62697.slice/crio-7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0 WatchSource:0}: Error finding container 7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0: Status 404 returned error can't find the container with id 7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0
Sep 18 20:18:07.774810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:07.774778    2566 generic.go:356] "Generic (PLEG): container finished" podID="21d1d885-79ec-4a07-ba11-07d49cd62697" containerID="587e9a0e627adb3df032bb996aa56d66adbc6415e3f77ba6a2905089ba3b397f" exitCode=0
Sep 18 20:18:07.775171 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:07.774864    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp" event={"ID":"21d1d885-79ec-4a07-ba11-07d49cd62697","Type":"ContainerDied","Data":"587e9a0e627adb3df032bb996aa56d66adbc6415e3f77ba6a2905089ba3b397f"}
Sep 18 20:18:07.775171 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:07.774915    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp" event={"ID":"21d1d885-79ec-4a07-ba11-07d49cd62697","Type":"ContainerStarted","Data":"7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0"}
Sep 18 20:18:08.918025 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:08.917997    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:08.998563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:08.998535    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca\") pod \"21d1d885-79ec-4a07-ba11-07d49cd62697\" (UID: \"21d1d885-79ec-4a07-ba11-07d49cd62697\") "
Sep 18 20:18:08.998864 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:08.998843    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "21d1d885-79ec-4a07-ba11-07d49cd62697" (UID: "21d1d885-79ec-4a07-ba11-07d49cd62697"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:09.099947 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:09.099881    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/21d1d885-79ec-4a07-ba11-07d49cd62697-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:09.793381 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:09.793352    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"
Sep 18 20:18:09.793548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:09.793382    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp" event={"ID":"21d1d885-79ec-4a07-ba11-07d49cd62697","Type":"ContainerDied","Data":"7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0"}
Sep 18 20:18:09.793548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:09.793430    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7c0c0f1bf835c82435c7a5e085131a08766e7c1045f1a9949428189c3332ced0"
Sep 18 20:18:13.220852 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.220820    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:18:13.221370 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.221089    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" podUID="1a353a62-c591-44e1-84af-eb63da475f21" containerName="payload-processing" containerID="cri-o://e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9" gracePeriod=30
Sep 18 20:18:13.230428 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.230402    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:18:13.230637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.230612    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" podUID="007760ab-2829-4cb9-8712-d0434e5a9d17" containerName="payload-pre-processing" containerID="cri-o://bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615" gracePeriod=30
Sep 18 20:18:13.408157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.408069    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"]
Sep 18 20:18:13.411281 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.411251    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-migrate-6d00b9e3-4shhp"]
Sep 18 20:18:13.516936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.516912    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:18:13.534164 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.534146    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:18:13.648892 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.648863    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hkll\" (UniqueName: \"kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll\") pod \"1a353a62-c591-44e1-84af-eb63da475f21\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") "
Sep 18 20:18:13.649044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.648908    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume\") pod \"007760ab-2829-4cb9-8712-d0434e5a9d17\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") "
Sep 18 20:18:13.649044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.648947    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hsjmc\" (UniqueName: \"kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc\") pod \"007760ab-2829-4cb9-8712-d0434e5a9d17\" (UID: \"007760ab-2829-4cb9-8712-d0434e5a9d17\") "
Sep 18 20:18:13.649169 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.649097    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume\") pod \"1a353a62-c591-44e1-84af-eb63da475f21\" (UID: \"1a353a62-c591-44e1-84af-eb63da475f21\") "
Sep 18 20:18:13.649328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.649303    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "007760ab-2829-4cb9-8712-d0434e5a9d17" (UID: "007760ab-2829-4cb9-8712-d0434e5a9d17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:13.649445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.649422    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1a353a62-c591-44e1-84af-eb63da475f21" (UID: "1a353a62-c591-44e1-84af-eb63da475f21"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:13.649601 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.649583    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1a353a62-c591-44e1-84af-eb63da475f21-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.649660 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.649607    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/007760ab-2829-4cb9-8712-d0434e5a9d17-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.650960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.650941    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll" (OuterVolumeSpecName: "kube-api-access-2hkll") pod "1a353a62-c591-44e1-84af-eb63da475f21" (UID: "1a353a62-c591-44e1-84af-eb63da475f21"). InnerVolumeSpecName "kube-api-access-2hkll". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:13.651008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.650971    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc" (OuterVolumeSpecName: "kube-api-access-hsjmc") pod "007760ab-2829-4cb9-8712-d0434e5a9d17" (UID: "007760ab-2829-4cb9-8712-d0434e5a9d17"). InnerVolumeSpecName "kube-api-access-hsjmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:18:13.750548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.750466    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hkll\" (UniqueName: \"kubernetes.io/projected/1a353a62-c591-44e1-84af-eb63da475f21-kube-api-access-2hkll\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.750548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.750494    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hsjmc\" (UniqueName: \"kubernetes.io/projected/007760ab-2829-4cb9-8712-d0434e5a9d17-kube-api-access-hsjmc\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:13.816382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.816345    2566 generic.go:356] "Generic (PLEG): container finished" podID="1a353a62-c591-44e1-84af-eb63da475f21" containerID="e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9" exitCode=0
Sep 18 20:18:13.816549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.816389    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"
Sep 18 20:18:13.816549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.816431    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" event={"ID":"1a353a62-c591-44e1-84af-eb63da475f21","Type":"ContainerDied","Data":"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"}
Sep 18 20:18:13.816549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.816518    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm" event={"ID":"1a353a62-c591-44e1-84af-eb63da475f21","Type":"ContainerDied","Data":"403a888ae03500c900066978d158887ab7b861e48135cb4873068f5e90f0d489"}
Sep 18 20:18:13.816549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.816548    2566 scope.go:117] "RemoveContainer" containerID="e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"
Sep 18 20:18:13.817890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.817870    2566 generic.go:356] "Generic (PLEG): container finished" podID="007760ab-2829-4cb9-8712-d0434e5a9d17" containerID="bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615" exitCode=0
Sep 18 20:18:13.818016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.817905    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"
Sep 18 20:18:13.818016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.817917    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" event={"ID":"007760ab-2829-4cb9-8712-d0434e5a9d17","Type":"ContainerDied","Data":"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"}
Sep 18 20:18:13.818016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.817963    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml" event={"ID":"007760ab-2829-4cb9-8712-d0434e5a9d17","Type":"ContainerDied","Data":"ffb14bcc7d43643739d89c74381589332195142e202c6291577d7866ed64705f"}
Sep 18 20:18:13.826231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.826214    2566 scope.go:117] "RemoveContainer" containerID="e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"
Sep 18 20:18:13.826524 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:18:13.826508    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9\": container with ID starting with e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9 not found: ID does not exist" containerID="e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"
Sep 18 20:18:13.826572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.826531    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9"} err="failed to get container status \"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9\": rpc error: code = NotFound desc = could not find container \"e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9\": container with ID starting with e4833b905601fb3642fd008e6b6558af11f09ae8c085cbb6fb5de4bf43339df9 not found: ID does not exist"
Sep 18 20:18:13.826572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.826546    2566 scope.go:117] "RemoveContainer" containerID="bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"
Sep 18 20:18:13.834707 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.834690    2566 scope.go:117] "RemoveContainer" containerID="bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"
Sep 18 20:18:13.834951 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:18:13.834936    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615\": container with ID starting with bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615 not found: ID does not exist" containerID="bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"
Sep 18 20:18:13.835004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.834958    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615"} err="failed to get container status \"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615\": rpc error: code = NotFound desc = could not find container \"bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615\": container with ID starting with bb0c9ec98555afbd85671a371f0d0031c8052a24efab4d9ad6cc59cc1e337615 not found: ID does not exist"
Sep 18 20:18:13.842363 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.842321    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:18:13.844719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.844700    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-migrate-6d00b9e3-695f4c6fd6-5zfhm"]
Sep 18 20:18:13.853857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.853837    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:18:13.863379 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:13.863359    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-migrate-6d00b9e3-6b6cf7b5f4-4cbml"]
Sep 18 20:18:14.081598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.081507    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="007760ab-2829-4cb9-8712-d0434e5a9d17" path="/var/lib/kubelet/pods/007760ab-2829-4cb9-8712-d0434e5a9d17/volumes"
Sep 18 20:18:14.081885 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.081871    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a353a62-c591-44e1-84af-eb63da475f21" path="/var/lib/kubelet/pods/1a353a62-c591-44e1-84af-eb63da475f21/volumes"
Sep 18 20:18:14.082207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.082193    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="21d1d885-79ec-4a07-ba11-07d49cd62697" path="/var/lib/kubelet/pods/21d1d885-79ec-4a07-ba11-07d49cd62697/volumes"
Sep 18 20:18:14.337390 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337311    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:18:14.337767 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337755    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="007760ab-2829-4cb9-8712-d0434e5a9d17" containerName="payload-pre-processing"
Sep 18 20:18:14.337811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337769    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="007760ab-2829-4cb9-8712-d0434e5a9d17" containerName="payload-pre-processing"
Sep 18 20:18:14.337811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337790    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a353a62-c591-44e1-84af-eb63da475f21" containerName="payload-processing"
Sep 18 20:18:14.337811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337797    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a353a62-c591-44e1-84af-eb63da475f21" containerName="payload-processing"
Sep 18 20:18:14.337811 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337807    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="21d1d885-79ec-4a07-ba11-07d49cd62697" containerName="revoke-keys"
Sep 18 20:18:14.337935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337812    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="21d1d885-79ec-4a07-ba11-07d49cd62697" containerName="revoke-keys"
Sep 18 20:18:14.337935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337889    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="007760ab-2829-4cb9-8712-d0434e5a9d17" containerName="payload-pre-processing"
Sep 18 20:18:14.337935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337897    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a353a62-c591-44e1-84af-eb63da475f21" containerName="payload-processing"
Sep 18 20:18:14.337935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.337902    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="21d1d885-79ec-4a07-ba11-07d49cd62697" containerName="revoke-keys"
Sep 18 20:18:14.341292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.341269    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.344011 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.343990    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ait-f88d7657-openshift-default-dockercfg-n5ks9\""
Sep 18 20:18:14.358786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.358760    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:18:14.457982 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.457939    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.457982 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.457984    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458035    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458083    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458147    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458254    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458277    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ns7qh\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.458354 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.458314    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559620    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559664    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559768    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ns7qh\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559813    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559853    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.559918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559882    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559936    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.559970    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.560252    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.560268    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.560333    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.560466    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.560591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.560571    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.562218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.562194    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.562348 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.562329    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.571650 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.571623    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.571786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.571685    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ns7qh\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh\") pod \"e2e-ait-f88d7657-openshift-default-648684c7fd-qn275\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") " pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.651777 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.651708    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:14.929067 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.929030    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:18:14.930904 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:18:14.930878    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f25d6be_9bef_486a_9b15_74d0221a3033.slice/crio-360f19bb7be246d71444f6cbdf39bddbbc8d5682d945684ccb26e1c6d096b934 WatchSource:0}: Error finding container 360f19bb7be246d71444f6cbdf39bddbbc8d5682d945684ccb26e1c6d096b934: Status 404 returned error can't find the container with id 360f19bb7be246d71444f6cbdf39bddbbc8d5682d945684ccb26e1c6d096b934
Sep 18 20:18:14.933257 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.933223    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:14.933339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.933290    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:14.933339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:14.933320    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:15.833068 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:15.833023    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerStarted","Data":"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"}
Sep 18 20:18:15.833446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:15.833093    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerStarted","Data":"360f19bb7be246d71444f6cbdf39bddbbc8d5682d945684ccb26e1c6d096b934"}
Sep 18 20:18:15.856947 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:15.856901    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podStartSLOduration=1.8568863370000002 podStartE2EDuration="1.856886337s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:15.85524749 +0000 UTC m=+1039.984549446" watchObservedRunningTime="2026-09-18 20:18:15.856886337 +0000 UTC m=+1039.986188288"
Sep 18 20:18:16.652664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:16.652620    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:16.658542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:16.658518    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:16.839196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:16.839162    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:16.840180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:16.840162    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:42.001534 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.001483    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerID="dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9" exitCode=0
Sep 18 20:18:42.002108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.001615    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerDied","Data":"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"}
Sep 18 20:18:42.002294 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.002233    2566 scope.go:117] "RemoveContainer" containerID="dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"
Sep 18 20:18:42.003801 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.003765    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:42.003908 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.003886    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:42.003971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:42.003934    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:18:43.009013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:43.008978    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerStarted","Data":"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"}
Sep 18 20:18:43.653365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:43.653334    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:43.653760 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:43.653733    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 18 20:18:43.653954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:43.653929    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 18 20:18:44.652451 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:44.652407    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 18 20:18:44.652944 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:44.652475    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 18 20:18:45.652774 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:45.652730    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 18 20:18:45.653267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:45.652838    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 18 20:18:46.652717 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:46.652679    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 18 20:18:46.653014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:46.652982    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 18 20:18:46.839571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:46.839529    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:47.653251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:47.653209    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused" start-of-body=
Sep 18 20:18:47.653825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:47.653794    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.60:15021/healthz/ready\": dial tcp 10.132.0.60:15021: connect: connection refused"
Sep 18 20:18:48.018448 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.018405    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"]
Sep 18 20:18:48.022250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.022221    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:48.024706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.024678    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:18:48.024858 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.024758    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:18:48.029857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.029828    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"]
Sep 18 20:18:48.126027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.125996    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f88d7657-27kdl\" (UID: \"667fe1be-8602-4f2b-8e53-916a4d971f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:48.227256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.227218    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f88d7657-27kdl\" (UID: \"667fe1be-8602-4f2b-8e53-916a4d971f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:48.227838 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.227818    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ait-f88d7657-27kdl\" (UID: \"667fe1be-8602-4f2b-8e53-916a4d971f95\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:48.337946 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.337856    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:48.607592 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.607569    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"]
Sep 18 20:18:48.609839 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:18:48.609808    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod667fe1be_8602_4f2b_8e53_916a4d971f95.slice/crio-bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916 WatchSource:0}: Error finding container bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916: Status 404 returned error can't find the container with id bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916
Sep 18 20:18:48.655994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.655971    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:48.656973 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:48.656954    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:18:49.054525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:49.054493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl" event={"ID":"667fe1be-8602-4f2b-8e53-916a4d971f95","Type":"ContainerStarted","Data":"1868cc67deaabb9c47413ad2c40eb050b7cad0ddf34a7715b37246e11fe2621c"}
Sep 18 20:18:49.054710 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:49.054543    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl" event={"ID":"667fe1be-8602-4f2b-8e53-916a4d971f95","Type":"ContainerStarted","Data":"bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916"}
Sep 18 20:18:49.082619 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:49.082571    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl" podStartSLOduration=1.082554367 podStartE2EDuration="1.082554367s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:18:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:18:49.078189596 +0000 UTC m=+1073.207491546" watchObservedRunningTime="2026-09-18 20:18:49.082554367 +0000 UTC m=+1073.211856317"
Sep 18 20:18:50.062081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:50.062023    2566 generic.go:356] "Generic (PLEG): container finished" podID="667fe1be-8602-4f2b-8e53-916a4d971f95" containerID="1868cc67deaabb9c47413ad2c40eb050b7cad0ddf34a7715b37246e11fe2621c" exitCode=0
Sep 18 20:18:50.062476 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:50.062120    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl" event={"ID":"667fe1be-8602-4f2b-8e53-916a4d971f95","Type":"ContainerDied","Data":"1868cc67deaabb9c47413ad2c40eb050b7cad0ddf34a7715b37246e11fe2621c"}
Sep 18 20:18:51.219456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:51.219429    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:51.361041 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:51.360954    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca\") pod \"667fe1be-8602-4f2b-8e53-916a4d971f95\" (UID: \"667fe1be-8602-4f2b-8e53-916a4d971f95\") "
Sep 18 20:18:51.361303 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:51.361282    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "667fe1be-8602-4f2b-8e53-916a4d971f95" (UID: "667fe1be-8602-4f2b-8e53-916a4d971f95"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:18:51.462168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:51.462128    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/667fe1be-8602-4f2b-8e53-916a4d971f95-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:18:52.076152 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:52.076125    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"
Sep 18 20:18:52.083118 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:52.083079    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl" event={"ID":"667fe1be-8602-4f2b-8e53-916a4d971f95","Type":"ContainerDied","Data":"bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916"}
Sep 18 20:18:52.083252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:52.083126    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bb37f1dc5ef37f88d4de8bbadabec326d8cf6a47e126c4e209a609c0299db916"
Sep 18 20:18:59.237161 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:59.236310    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"]
Sep 18 20:18:59.245503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:18:59.245471    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ait-f88d7657-27kdl"]
Sep 18 20:19:00.080991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:00.080956    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="667fe1be-8602-4f2b-8e53-916a4d971f95" path="/var/lib/kubelet/pods/667fe1be-8602-4f2b-8e53-916a4d971f95/volumes"
Sep 18 20:19:00.355384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:00.355298    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:19:00.355840 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:00.355555    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" containerID="cri-o://94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31" gracePeriod=30
Sep 18 20:19:03.659117 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:03.659046    2566 patch_prober.go:28] interesting pod/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:19:03.659560 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:03.659152    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:19:05.616716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.616692    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:19:05.808311 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808282    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808336    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808387    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808457    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808495    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808526    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808558    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808624    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ns7qh\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808669    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket\") pod \"3f25d6be-9bef-486a-9b15-74d0221a3033\" (UID: \"3f25d6be-9bef-486a-9b15-74d0221a3033\") "
Sep 18 20:19:05.808844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808719    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data" (OuterVolumeSpecName: "istio-data") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:05.808844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808735    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:05.808844 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808728    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:19:05.809071 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.808852    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:05.809152 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.809086    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:05.809211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.809166    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3f25d6be-9bef-486a-9b15-74d0221a3033-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.809211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.809188    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.809211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.809202    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.809339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.809223    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.810871 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.810836    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:19:05.811071 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.811039    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh" (OuterVolumeSpecName: "kube-api-access-ns7qh") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "kube-api-access-ns7qh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:05.811182 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.811042    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:19:05.811182 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.811087    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token" (OuterVolumeSpecName: "istio-token") pod "3f25d6be-9bef-486a-9b15-74d0221a3033" (UID: "3f25d6be-9bef-486a-9b15-74d0221a3033"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:05.910297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.910263    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.910297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.910289    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.910297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.910303    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ns7qh\" (UniqueName: \"kubernetes.io/projected/3f25d6be-9bef-486a-9b15-74d0221a3033-kube-api-access-ns7qh\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.910515 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.910316    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:05.910515 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:05.910328    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3f25d6be-9bef-486a-9b15-74d0221a3033-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:06.156645 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.156607    2566 generic.go:356] "Generic (PLEG): container finished" podID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerID="94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31" exitCode=0
Sep 18 20:19:06.156812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.156660    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"
Sep 18 20:19:06.156812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.156695    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerDied","Data":"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"}
Sep 18 20:19:06.156812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.156760    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275" event={"ID":"3f25d6be-9bef-486a-9b15-74d0221a3033","Type":"ContainerDied","Data":"360f19bb7be246d71444f6cbdf39bddbbc8d5682d945684ccb26e1c6d096b934"}
Sep 18 20:19:06.156812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.156790    2566 scope.go:117] "RemoveContainer" containerID="94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"
Sep 18 20:19:06.166236 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.166220    2566 scope.go:117] "RemoveContainer" containerID="dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"
Sep 18 20:19:06.174538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.174520    2566 scope.go:117] "RemoveContainer" containerID="94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"
Sep 18 20:19:06.174817 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:19:06.174797    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31\": container with ID starting with 94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31 not found: ID does not exist" containerID="94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"
Sep 18 20:19:06.174884 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.174826    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31"} err="failed to get container status \"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31\": rpc error: code = NotFound desc = could not find container \"94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31\": container with ID starting with 94bc41189b5e06d560baf8f9f3a15c7f0ad276f5a2b69eb547539a509f678a31 not found: ID does not exist"
Sep 18 20:19:06.174884 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.174842    2566 scope.go:117] "RemoveContainer" containerID="dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"
Sep 18 20:19:06.175100 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:19:06.175082    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9\": container with ID starting with dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9 not found: ID does not exist" containerID="dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"
Sep 18 20:19:06.175175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.175104    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9"} err="failed to get container status \"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9\": rpc error: code = NotFound desc = could not find container \"dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9\": container with ID starting with dd70588a7979121e3a4943e563c78edcaac565dbd4ae7deab0cbdcf740bd32b9 not found: ID does not exist"
Sep 18 20:19:06.175175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.175135    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:19:06.178652 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:06.178633    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ait-f88d7657-openshift-default-648684c7fd-qn275"]
Sep 18 20:19:07.793224 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793190    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:07.793659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793644    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.793727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793662    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.793727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793688    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="667fe1be-8602-4f2b-8e53-916a4d971f95" containerName="revoke-keys"
Sep 18 20:19:07.793727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793694    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="667fe1be-8602-4f2b-8e53-916a4d971f95" containerName="revoke-keys"
Sep 18 20:19:07.793823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793761    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.793823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793768    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="667fe1be-8602-4f2b-8e53-916a4d971f95" containerName="revoke-keys"
Sep 18 20:19:07.793823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.793778    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.798480 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.798465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.800839 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.800821    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-derive-6e56ebc0-dockercfg-zgm9w\""
Sep 18 20:19:07.801220 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.801205    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-derive-6e56ebc0\""
Sep 18 20:19:07.808631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.808609    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:07.824851 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.824828    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxffx\" (UniqueName: \"kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.824936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.824887    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.832579 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.832561    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:07.832988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.832975    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.832988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.832989    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" containerName="istio-proxy"
Sep 18 20:19:07.835946 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.835931    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:07.848261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.848240    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:07.926288 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.926249    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48drk\" (UniqueName: \"kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:07.926477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.926309    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxffx\" (UniqueName: \"kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.926477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.926391    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:07.926477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.926421    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.927066 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.927037    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:07.936483 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:07.936460    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxffx\" (UniqueName: \"kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx\") pod \"payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") " pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:08.026832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.026806    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:08.026964 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.026893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48drk\" (UniqueName: \"kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:08.027499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.027480    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:08.034753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.034732    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48drk\" (UniqueName: \"kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk\") pod \"payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") " pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:08.081434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.081376    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3f25d6be-9bef-486a-9b15-74d0221a3033" path="/var/lib/kubelet/pods/3f25d6be-9bef-486a-9b15-74d0221a3033/volumes"
Sep 18 20:19:08.109644 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.109624    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:08.148622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.148533    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:08.393373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.393347    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:08.394817 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:19:08.394789    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596ddee1_b840_4437_ac31_507c388b2d93.slice/crio-e99d68e8c00dd7f87405b1ded4bf9e99d02db2a0710010a63bff1079ad155036 WatchSource:0}: Error finding container e99d68e8c00dd7f87405b1ded4bf9e99d02db2a0710010a63bff1079ad155036: Status 404 returned error can't find the container with id e99d68e8c00dd7f87405b1ded4bf9e99d02db2a0710010a63bff1079ad155036
Sep 18 20:19:08.439840 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:08.439816    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:08.441589 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:19:08.441555    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9b53878_6d22_4195_9dad_5ad9dde7da17.slice/crio-1f65abcf1189fcf246be0546fa3ed2c894f6b146cf0fab14a2870f0b6f0e4045 WatchSource:0}: Error finding container 1f65abcf1189fcf246be0546fa3ed2c894f6b146cf0fab14a2870f0b6f0e4045: Status 404 returned error can't find the container with id 1f65abcf1189fcf246be0546fa3ed2c894f6b146cf0fab14a2870f0b6f0e4045
Sep 18 20:19:09.183740 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.183673    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" event={"ID":"f9b53878-6d22-4195-9dad-5ad9dde7da17","Type":"ContainerStarted","Data":"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"}
Sep 18 20:19:09.184217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.183761    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" event={"ID":"f9b53878-6d22-4195-9dad-5ad9dde7da17","Type":"ContainerStarted","Data":"1f65abcf1189fcf246be0546fa3ed2c894f6b146cf0fab14a2870f0b6f0e4045"}
Sep 18 20:19:09.184217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.183793    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:09.185436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.185408    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" event={"ID":"596ddee1-b840-4437-ac31-507c388b2d93","Type":"ContainerStarted","Data":"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"}
Sep 18 20:19:09.185580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.185454    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" event={"ID":"596ddee1-b840-4437-ac31-507c388b2d93","Type":"ContainerStarted","Data":"e99d68e8c00dd7f87405b1ded4bf9e99d02db2a0710010a63bff1079ad155036"}
Sep 18 20:19:09.185580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.185523    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:09.201833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.201783    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" podStartSLOduration=2.201763518 podStartE2EDuration="2.201763518s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:09.199461215 +0000 UTC m=+1093.328763166" watchObservedRunningTime="2026-09-18 20:19:09.201763518 +0000 UTC m=+1093.331065473"
Sep 18 20:19:09.216395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:09.216354    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" podStartSLOduration=2.216340703 podStartE2EDuration="2.216340703s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:19:09.21438487 +0000 UTC m=+1093.343686841" watchObservedRunningTime="2026-09-18 20:19:09.216340703 +0000 UTC m=+1093.345642657"
Sep 18 20:19:20.192699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:20.192662    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:20.193079 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:20.192731    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:33.801374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.801335    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"]
Sep 18 20:19:33.804972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.804950    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:33.807245 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.807213    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:19:33.807404 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.807295    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:19:33.811926 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.811895    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"]
Sep 18 20:19:33.879945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.879903    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb\" (UID: \"9d076de2-580c-48cf-8150-fe521feb0b9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:33.980961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.980931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb\" (UID: \"9d076de2-580c-48cf-8150-fe521feb0b9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:33.981570 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:33.981547    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb\" (UID: \"9d076de2-580c-48cf-8150-fe521feb0b9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:34.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:34.118471    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:34.387450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:34.387429    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"]
Sep 18 20:19:34.389377 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:19:34.389352    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d076de2_580c_48cf_8150_fe521feb0b9f.slice/crio-49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f WatchSource:0}: Error finding container 49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f: Status 404 returned error can't find the container with id 49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f
Sep 18 20:19:35.334168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:35.334134    2566 generic.go:356] "Generic (PLEG): container finished" podID="9d076de2-580c-48cf-8150-fe521feb0b9f" containerID="297af1590a3f24d1e740301ca83d13cc17f48b8121c858941b90e1e5f267f66f" exitCode=0
Sep 18 20:19:35.334595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:35.334224    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb" event={"ID":"9d076de2-580c-48cf-8150-fe521feb0b9f","Type":"ContainerDied","Data":"297af1590a3f24d1e740301ca83d13cc17f48b8121c858941b90e1e5f267f66f"}
Sep 18 20:19:35.334595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:35.334279    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb" event={"ID":"9d076de2-580c-48cf-8150-fe521feb0b9f","Type":"ContainerStarted","Data":"49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f"}
Sep 18 20:19:36.485431 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:36.485409    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:36.608739 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:36.608673    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca\") pod \"9d076de2-580c-48cf-8150-fe521feb0b9f\" (UID: \"9d076de2-580c-48cf-8150-fe521feb0b9f\") "
Sep 18 20:19:36.609031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:36.609006    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "9d076de2-580c-48cf-8150-fe521feb0b9f" (UID: "9d076de2-580c-48cf-8150-fe521feb0b9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:19:36.709742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:36.709713    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/9d076de2-580c-48cf-8150-fe521feb0b9f-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:37.350371 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:37.350339    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"
Sep 18 20:19:37.350548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:37.350369    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb" event={"ID":"9d076de2-580c-48cf-8150-fe521feb0b9f","Type":"ContainerDied","Data":"49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f"}
Sep 18 20:19:37.350548 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:37.350416    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="49920983944cce2d765898f281c7f4343fcfe71c9b75efade25f735ad4fa505f"
Sep 18 20:19:45.803554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.803522    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:45.804196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.804012    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" podUID="f9b53878-6d22-4195-9dad-5ad9dde7da17" containerName="payload-processing" containerID="cri-o://1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29" gracePeriod=30
Sep 18 20:19:45.809388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.809353    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:45.809650 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.809604    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" podUID="596ddee1-b840-4437-ac31-507c388b2d93" containerName="payload-pre-processing" containerID="cri-o://96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70" gracePeriod=30
Sep 18 20:19:45.990214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.990167    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"]
Sep 18 20:19:45.992011 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:45.991986    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-derive-6e56ebc0-tfmfb"]
Sep 18 20:19:46.088772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.088559    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d076de2-580c-48cf-8150-fe521feb0b9f" path="/var/lib/kubelet/pods/9d076de2-580c-48cf-8150-fe521feb0b9f/volumes"
Sep 18 20:19:46.090611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.090590    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:46.117979 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.117739    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:46.202680 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.202641    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48drk\" (UniqueName: \"kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk\") pod \"f9b53878-6d22-4195-9dad-5ad9dde7da17\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") "
Sep 18 20:19:46.202863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.202784    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume\") pod \"f9b53878-6d22-4195-9dad-5ad9dde7da17\" (UID: \"f9b53878-6d22-4195-9dad-5ad9dde7da17\") "
Sep 18 20:19:46.203140 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.203113    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f9b53878-6d22-4195-9dad-5ad9dde7da17" (UID: "f9b53878-6d22-4195-9dad-5ad9dde7da17"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:19:46.203261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.203188    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f9b53878-6d22-4195-9dad-5ad9dde7da17-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:46.204812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.204795    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk" (OuterVolumeSpecName: "kube-api-access-48drk") pod "f9b53878-6d22-4195-9dad-5ad9dde7da17" (UID: "f9b53878-6d22-4195-9dad-5ad9dde7da17"). InnerVolumeSpecName "kube-api-access-48drk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:46.304284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.304244    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume\") pod \"596ddee1-b840-4437-ac31-507c388b2d93\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") "
Sep 18 20:19:46.304477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.304372    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxffx\" (UniqueName: \"kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx\") pod \"596ddee1-b840-4437-ac31-507c388b2d93\" (UID: \"596ddee1-b840-4437-ac31-507c388b2d93\") "
Sep 18 20:19:46.304671 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.304643    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "596ddee1-b840-4437-ac31-507c388b2d93" (UID: "596ddee1-b840-4437-ac31-507c388b2d93"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:19:46.304789 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.304767    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/596ddee1-b840-4437-ac31-507c388b2d93-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:46.304829 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.304794    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48drk\" (UniqueName: \"kubernetes.io/projected/f9b53878-6d22-4195-9dad-5ad9dde7da17-kube-api-access-48drk\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:46.306623 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.306555    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx" (OuterVolumeSpecName: "kube-api-access-mxffx") pod "596ddee1-b840-4437-ac31-507c388b2d93" (UID: "596ddee1-b840-4437-ac31-507c388b2d93"). InnerVolumeSpecName "kube-api-access-mxffx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:19:46.402975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.402936    2566 generic.go:356] "Generic (PLEG): container finished" podID="596ddee1-b840-4437-ac31-507c388b2d93" containerID="96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70" exitCode=0
Sep 18 20:19:46.403202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.402983    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"
Sep 18 20:19:46.403202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.403026    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" event={"ID":"596ddee1-b840-4437-ac31-507c388b2d93","Type":"ContainerDied","Data":"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"}
Sep 18 20:19:46.403202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.403110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl" event={"ID":"596ddee1-b840-4437-ac31-507c388b2d93","Type":"ContainerDied","Data":"e99d68e8c00dd7f87405b1ded4bf9e99d02db2a0710010a63bff1079ad155036"}
Sep 18 20:19:46.403202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.403144    2566 scope.go:117] "RemoveContainer" containerID="96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"
Sep 18 20:19:46.404724 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.404698    2566 generic.go:356] "Generic (PLEG): container finished" podID="f9b53878-6d22-4195-9dad-5ad9dde7da17" containerID="1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29" exitCode=0
Sep 18 20:19:46.404828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.404744    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"
Sep 18 20:19:46.404828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.404765    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" event={"ID":"f9b53878-6d22-4195-9dad-5ad9dde7da17","Type":"ContainerDied","Data":"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"}
Sep 18 20:19:46.404828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.404810    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr" event={"ID":"f9b53878-6d22-4195-9dad-5ad9dde7da17","Type":"ContainerDied","Data":"1f65abcf1189fcf246be0546fa3ed2c894f6b146cf0fab14a2870f0b6f0e4045"}
Sep 18 20:19:46.405400 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.405382    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxffx\" (UniqueName: \"kubernetes.io/projected/596ddee1-b840-4437-ac31-507c388b2d93-kube-api-access-mxffx\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:19:46.414373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.414357    2566 scope.go:117] "RemoveContainer" containerID="96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"
Sep 18 20:19:46.414666 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:19:46.414644    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70\": container with ID starting with 96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70 not found: ID does not exist" containerID="96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"
Sep 18 20:19:46.414787 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.414673    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70"} err="failed to get container status \"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70\": rpc error: code = NotFound desc = could not find container \"96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70\": container with ID starting with 96ea78adf6ee2168088ebb6d0165f615d043aa9f8b9f5791a281bcf20a059e70 not found: ID does not exist"
Sep 18 20:19:46.414787 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.414690    2566 scope.go:117] "RemoveContainer" containerID="1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"
Sep 18 20:19:46.423940 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.423921    2566 scope.go:117] "RemoveContainer" containerID="1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"
Sep 18 20:19:46.424223 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:19:46.424204    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29\": container with ID starting with 1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29 not found: ID does not exist" containerID="1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"
Sep 18 20:19:46.424299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.424228    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29"} err="failed to get container status \"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29\": rpc error: code = NotFound desc = could not find container \"1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29\": container with ID starting with 1158c07f0badfc3d9986f9553776706f57244b265008354acaf799a0bec49c29 not found: ID does not exist"
Sep 18 20:19:46.429623 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.429603    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:46.434724 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.434704    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-derive-6e56ebc0-74654f4958-rpksl"]
Sep 18 20:19:46.444858 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.444824    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:46.449238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:46.449216    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-derive-6e56ebc0-5fb9dbd55c-jrhbr"]
Sep 18 20:19:48.084469 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:48.084432    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="596ddee1-b840-4437-ac31-507c388b2d93" path="/var/lib/kubelet/pods/596ddee1-b840-4437-ac31-507c388b2d93/volumes"
Sep 18 20:19:48.085033 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:48.085004    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f9b53878-6d22-4195-9dad-5ad9dde7da17" path="/var/lib/kubelet/pods/f9b53878-6d22-4195-9dad-5ad9dde7da17/volumes"
Sep 18 20:19:59.904905 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.904861    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:19:59.905388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905344    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d076de2-580c-48cf-8150-fe521feb0b9f" containerName="revoke-keys"
Sep 18 20:19:59.905388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905358    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d076de2-580c-48cf-8150-fe521feb0b9f" containerName="revoke-keys"
Sep 18 20:19:59.905388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905371    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="596ddee1-b840-4437-ac31-507c388b2d93" containerName="payload-pre-processing"
Sep 18 20:19:59.905388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905378    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="596ddee1-b840-4437-ac31-507c388b2d93" containerName="payload-pre-processing"
Sep 18 20:19:59.905388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905391    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9b53878-6d22-4195-9dad-5ad9dde7da17" containerName="payload-processing"
Sep 18 20:19:59.905661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905398    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9b53878-6d22-4195-9dad-5ad9dde7da17" containerName="payload-processing"
Sep 18 20:19:59.905661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905480    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9b53878-6d22-4195-9dad-5ad9dde7da17" containerName="payload-processing"
Sep 18 20:19:59.905661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905486    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d076de2-580c-48cf-8150-fe521feb0b9f" containerName="revoke-keys"
Sep 18 20:19:59.905661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.905495    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="596ddee1-b840-4437-ac31-507c388b2d93" containerName="payload-pre-processing"
Sep 18 20:19:59.910237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.910218    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:19:59.913402 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.913378    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a85f8b\"/\"kube-root-ca.crt\""
Sep 18 20:19:59.913520 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.913501    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a85f8b\"/\"test-model-a85f8b-kserve-self-signed-certs\""
Sep 18 20:19:59.913576 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.913550    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-b-a85f8b\"/\"openshift-service-ca.crt\""
Sep 18 20:19:59.913631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.913594    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-b-a85f8b\"/\"default-dockercfg-ffgrl\""
Sep 18 20:19:59.916712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:19:59.916690    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:20:00.033928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.033878    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.034126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.033935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.034126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.033987    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7b6d\" (UniqueName: \"kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.034126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.034022    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.034126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.034044    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.034126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.034090    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134728 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134693    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7b6d\" (UniqueName: \"kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134728 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134734    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134757    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134777    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134845    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.134996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.134863    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.135252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.135226    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.135314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.135239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.135314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.135291    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.137339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.137316    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.137481 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.137460    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.142459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.142439    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7b6d\" (UniqueName: \"kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d\") pod \"test-model-a85f8b-kserve-66d77b8bd5-b74h6\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") " pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.221240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.221209    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:00.505221 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:00.505182    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:20:00.505946 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:20:00.505918    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podddc9f839_7a9e_44e5_8721_c649469805f1.slice/crio-5279c927e3f4445a4cfb7a9ce26d2d4cd5cc006745b4163283855c58036bb21d WatchSource:0}: Error finding container 5279c927e3f4445a4cfb7a9ce26d2d4cd5cc006745b4163283855c58036bb21d: Status 404 returned error can't find the container with id 5279c927e3f4445a4cfb7a9ce26d2d4cd5cc006745b4163283855c58036bb21d
Sep 18 20:20:01.502503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:01.502469    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" event={"ID":"ddc9f839-7a9e-44e5-8721-c649469805f1","Type":"ContainerStarted","Data":"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"}
Sep 18 20:20:01.502503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:01.502506    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" event={"ID":"ddc9f839-7a9e-44e5-8721-c649469805f1","Type":"ContainerStarted","Data":"5279c927e3f4445a4cfb7a9ce26d2d4cd5cc006745b4163283855c58036bb21d"}
Sep 18 20:20:01.519311 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:01.519266    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" podStartSLOduration=2.519250467 podStartE2EDuration="2.519250467s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:19:59 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:20:01.51823125 +0000 UTC m=+1145.647533202" watchObservedRunningTime="2026-09-18 20:20:01.519250467 +0000 UTC m=+1145.648552417"
Sep 18 20:20:10.221949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:10.221858    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:10.221949 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:10.221907    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:10.234502 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:10.234479    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:10.571887 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:10.571804    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:20:58.109821 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:58.109792    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:20:58.110388 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:20:58.110211    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:21:21.856012 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.855974    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:21:21.859864 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.859844    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.862928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.862902    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-802c6c2a-openshift-default-dockercfg-6wctc\""
Sep 18 20:21:21.872962 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.872939    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:21:21.962048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962009    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962176    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962214    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962273    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vrjpc\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962430    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962477    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:21.962512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:21.962506    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063165    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063239    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063255    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063275    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063336    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063394 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063362    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063407    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vrjpc\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063475    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063678    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063898    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063989 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.063989 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.063976    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.064306 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.064285    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.065934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.065899    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.066383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.066365    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.071395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.071372    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.071598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.071579    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vrjpc\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc\") pod \"e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") " pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.174310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.174228    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:22.472228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.472197    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:21:22.472365 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:21:22.472231    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d9506b_f54f_45f1_952e_b20e424cd113.slice/crio-32bd27100f268a06d97e022363266b89422a51fa97b01af3571223215c8225d8 WatchSource:0}: Error finding container 32bd27100f268a06d97e022363266b89422a51fa97b01af3571223215c8225d8: Status 404 returned error can't find the container with id 32bd27100f268a06d97e022363266b89422a51fa97b01af3571223215c8225d8
Sep 18 20:21:22.474985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.474952    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:22.475137 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.475020    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:22.475137 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:22.475048    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:23.009280 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:23.009230    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerStarted","Data":"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"}
Sep 18 20:21:23.009686 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:23.009298    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerStarted","Data":"32bd27100f268a06d97e022363266b89422a51fa97b01af3571223215c8225d8"}
Sep 18 20:21:23.035580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:23.035517    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podStartSLOduration=2.035496111 podStartE2EDuration="2.035496111s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:21 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:23.032179516 +0000 UTC m=+1227.161481466" watchObservedRunningTime="2026-09-18 20:21:23.035496111 +0000 UTC m=+1227.164798061"
Sep 18 20:21:23.175012 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:23.174968    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:23.181330 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:23.181291    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:24.015554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:24.015522    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:24.016384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:24.016364    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:21:28.081972 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.081941    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:21:28.085817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.085798    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.089433 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.089414    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-802c6c2a\""
Sep 18 20:21:28.089523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.089497    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-802c6c2a-dockercfg-h2lkm\""
Sep 18 20:21:28.094598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.094574    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:21:28.228967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.228932    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhq72\" (UniqueName: \"kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.229200 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.229041    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.330602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.330570    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.330784 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.330658    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhq72\" (UniqueName: \"kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.331347 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.331323    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.339531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.339467    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhq72\" (UniqueName: \"kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72\") pod \"payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.397934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.397898    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:28.673222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:28.673197    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:21:28.674999 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:21:28.674967    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5abcbfe7_b0b7_4de3_b0e0_90dc6a716ff1.slice/crio-3af697b5fbd60e4aac036a0aa70e8c4f4d4ed1bee1a69dc9bf76cb0ca9f75f4c WatchSource:0}: Error finding container 3af697b5fbd60e4aac036a0aa70e8c4f4d4ed1bee1a69dc9bf76cb0ca9f75f4c: Status 404 returned error can't find the container with id 3af697b5fbd60e4aac036a0aa70e8c4f4d4ed1bee1a69dc9bf76cb0ca9f75f4c
Sep 18 20:21:29.047355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:29.047317    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" event={"ID":"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1","Type":"ContainerStarted","Data":"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"}
Sep 18 20:21:29.047538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:29.047382    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" event={"ID":"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1","Type":"ContainerStarted","Data":"3af697b5fbd60e4aac036a0aa70e8c4f4d4ed1bee1a69dc9bf76cb0ca9f75f4c"}
Sep 18 20:21:29.047538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:29.047412    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:29.064523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:29.064470    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" podStartSLOduration=1.064455447 podStartE2EDuration="1.064455447s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:29.061883816 +0000 UTC m=+1233.191185771" watchObservedRunningTime="2026-09-18 20:21:29.064455447 +0000 UTC m=+1233.193757410"
Sep 18 20:21:40.056397 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:40.056368    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:21:57.015823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.015792    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"]
Sep 18 20:21:57.019552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.019533    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:21:57.021784 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.021764    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:21:57.021906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.021767    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:21:57.026825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.026800    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"]
Sep 18 20:21:57.118618 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.118592    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9\" (UID: \"16d45a3e-8315-4b40-afd1-b4003810a922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:21:57.219621 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.219599    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9\" (UID: \"16d45a3e-8315-4b40-afd1-b4003810a922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:21:57.220256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.220239    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9\" (UID: \"16d45a3e-8315-4b40-afd1-b4003810a922\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:21:57.332839 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.332777    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:21:57.619389 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:57.619363    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"]
Sep 18 20:21:57.621413 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:21:57.621387    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16d45a3e_8315_4b40_afd1_b4003810a922.slice/crio-75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a WatchSource:0}: Error finding container 75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a: Status 404 returned error can't find the container with id 75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a
Sep 18 20:21:58.216769 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.216690    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9" event={"ID":"16d45a3e-8315-4b40-afd1-b4003810a922","Type":"ContainerStarted","Data":"efb6f02f0153a6daed1badcdb3d7310f9bdd2111348f8cf024539ecb6584a9b4"}
Sep 18 20:21:58.216769 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.216754    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9" event={"ID":"16d45a3e-8315-4b40-afd1-b4003810a922","Type":"ContainerStarted","Data":"75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a"}
Sep 18 20:21:58.218377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.218353    2566 generic.go:356] "Generic (PLEG): container finished" podID="16d9506b-f54f-45f1-952e-b20e424cd113" containerID="8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf" exitCode=0
Sep 18 20:21:58.218481 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.218393    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerDied","Data":"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"}
Sep 18 20:21:58.218754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.218740    2566 scope.go:117] "RemoveContainer" containerID="8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"
Sep 18 20:21:58.219849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.219819    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:58.219934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.219885    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:58.219934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.219914    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:21:58.230514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:58.230463    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9" podStartSLOduration=2.230446911 podStartE2EDuration="2.230446911s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:21:58.230050631 +0000 UTC m=+1262.359352583" watchObservedRunningTime="2026-09-18 20:21:58.230446911 +0000 UTC m=+1262.359748866"
Sep 18 20:21:59.226384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:59.226349    2566 generic.go:356] "Generic (PLEG): container finished" podID="16d45a3e-8315-4b40-afd1-b4003810a922" containerID="efb6f02f0153a6daed1badcdb3d7310f9bdd2111348f8cf024539ecb6584a9b4" exitCode=0
Sep 18 20:21:59.226823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:59.226433    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9" event={"ID":"16d45a3e-8315-4b40-afd1-b4003810a922","Type":"ContainerDied","Data":"efb6f02f0153a6daed1badcdb3d7310f9bdd2111348f8cf024539ecb6584a9b4"}
Sep 18 20:21:59.228339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:21:59.228315    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerStarted","Data":"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"}
Sep 18 20:22:00.175030 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.174990    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:00.175273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.175145    2566 patch_prober.go:28] interesting pod/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 20:22:00.175273 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.175205    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 20:22:00.457135 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.443216    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:22:00.573231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.572484    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca\") pod \"16d45a3e-8315-4b40-afd1-b4003810a922\" (UID: \"16d45a3e-8315-4b40-afd1-b4003810a922\") "
Sep 18 20:22:00.573231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.573186    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "16d45a3e-8315-4b40-afd1-b4003810a922" (UID: "16d45a3e-8315-4b40-afd1-b4003810a922"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:00.673530 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:00.673493    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/16d45a3e-8315-4b40-afd1-b4003810a922-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:01.174797 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:01.174696    2566 patch_prober.go:28] interesting pod/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 20:22:01.174797 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:01.174750    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 20:22:01.244754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:01.244710    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9" event={"ID":"16d45a3e-8315-4b40-afd1-b4003810a922","Type":"ContainerDied","Data":"75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a"}
Sep 18 20:22:01.244954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:01.244782    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="75493399186a50e0056b82837b16df015cb789e7fb5112fb197d43d62def0f1a"
Sep 18 20:22:01.244954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:01.244854    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"
Sep 18 20:22:02.174873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.174838    2566 patch_prober.go:28] interesting pod/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 20:22:02.175265 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.174892    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 20:22:02.382803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.382736    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:22:02.496374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.496342    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"]
Sep 18 20:22:02.496853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.496840    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d45a3e-8315-4b40-afd1-b4003810a922" containerName="revoke-keys"
Sep 18 20:22:02.496901 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.496858    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d45a3e-8315-4b40-afd1-b4003810a922" containerName="revoke-keys"
Sep 18 20:22:02.496980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.496971    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d45a3e-8315-4b40-afd1-b4003810a922" containerName="revoke-keys"
Sep 18 20:22:02.499360 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.499343    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:02.501681 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.501657    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:22:02.501832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.501709    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:22:02.506403 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.506379    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"]
Sep 18 20:22:02.593163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.593122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v\" (UID: \"37218b7b-fed4-444d-a743-b4d7c66b33f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:02.694006 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.693971    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v\" (UID: \"37218b7b-fed4-444d-a743-b4d7c66b33f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:02.694638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.694612    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v\" (UID: \"37218b7b-fed4-444d-a743-b4d7c66b33f0\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:02.810922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:02.810833    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:03.104346 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:03.104323    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"]
Sep 18 20:22:03.175523 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:03.175499    2566 patch_prober.go:28] interesting pod/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 20:22:03.175850 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:03.175545    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 20:22:03.265470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:03.265435    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v" event={"ID":"37218b7b-fed4-444d-a743-b4d7c66b33f0","Type":"ContainerStarted","Data":"50429a0606aa732b2723aae41c32bbd955db38d8cf83b8725d2d7f4750f4a016"}
Sep 18 20:22:04.174771 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:04.174716    2566 patch_prober.go:28] interesting pod/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused" start-of-body=
Sep 18 20:22:04.174935 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:04.174787    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.66:15021/healthz/ready\": dial tcp 10.132.0.66:15021: connect: connection refused"
Sep 18 20:22:04.274297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:04.274254    2566 generic.go:356] "Generic (PLEG): container finished" podID="37218b7b-fed4-444d-a743-b4d7c66b33f0" containerID="7beb072ab58b96ac575ce75e77446283746bdf9d6599acb64fa2369ddd469ce4" exitCode=0
Sep 18 20:22:04.274698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:04.274339    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v" event={"ID":"37218b7b-fed4-444d-a743-b4d7c66b33f0","Type":"ContainerDied","Data":"7beb072ab58b96ac575ce75e77446283746bdf9d6599acb64fa2369ddd469ce4"}
Sep 18 20:22:05.179328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.179298    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:05.179596 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.179574    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:05.180261 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.180245    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:05.429505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.429444    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:05.519162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.519132    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca\") pod \"37218b7b-fed4-444d-a743-b4d7c66b33f0\" (UID: \"37218b7b-fed4-444d-a743-b4d7c66b33f0\") "
Sep 18 20:22:05.519462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.519440    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "37218b7b-fed4-444d-a743-b4d7c66b33f0" (UID: "37218b7b-fed4-444d-a743-b4d7c66b33f0"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:05.620632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:05.620599    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/37218b7b-fed4-444d-a743-b4d7c66b33f0-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:06.295521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:06.295493    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"
Sep 18 20:22:06.295715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:06.295528    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v" event={"ID":"37218b7b-fed4-444d-a743-b4d7c66b33f0","Type":"ContainerDied","Data":"50429a0606aa732b2723aae41c32bbd955db38d8cf83b8725d2d7f4750f4a016"}
Sep 18 20:22:06.295715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:06.295572    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="50429a0606aa732b2723aae41c32bbd955db38d8cf83b8725d2d7f4750f4a016"
Sep 18 20:22:07.616663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.616631    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:22:07.617739 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.616857    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" podUID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" containerName="payload-pre-processing" containerID="cri-o://9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5" gracePeriod=30
Sep 18 20:22:07.794668 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.794633    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"]
Sep 18 20:22:07.798470 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.798444    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-802c6c2a-sxsn9"]
Sep 18 20:22:07.907339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.907309    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:22:07.952594 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.952555    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhq72\" (UniqueName: \"kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72\") pod \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") "
Sep 18 20:22:07.952763 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.952643    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume\") pod \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\" (UID: \"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1\") "
Sep 18 20:22:07.953110 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.953073    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" (UID: "5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:07.954778 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:07.954746    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72" (OuterVolumeSpecName: "kube-api-access-dhq72") pod "5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" (UID: "5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1"). InnerVolumeSpecName "kube-api-access-dhq72". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:08.053563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.053532    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhq72\" (UniqueName: \"kubernetes.io/projected/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-kube-api-access-dhq72\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:08.053563 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.053560    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:08.081925 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.081898    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d45a3e-8315-4b40-afd1-b4003810a922" path="/var/lib/kubelet/pods/16d45a3e-8315-4b40-afd1-b4003810a922/volumes"
Sep 18 20:22:08.308509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.308480    2566 generic.go:356] "Generic (PLEG): container finished" podID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" containerID="9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5" exitCode=0
Sep 18 20:22:08.308638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.308526    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"
Sep 18 20:22:08.308638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.308563    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" event={"ID":"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1","Type":"ContainerDied","Data":"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"}
Sep 18 20:22:08.308638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.308610    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87" event={"ID":"5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1","Type":"ContainerDied","Data":"3af697b5fbd60e4aac036a0aa70e8c4f4d4ed1bee1a69dc9bf76cb0ca9f75f4c"}
Sep 18 20:22:08.308638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.308634    2566 scope.go:117] "RemoveContainer" containerID="9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"
Sep 18 20:22:08.317791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.317772    2566 scope.go:117] "RemoveContainer" containerID="9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"
Sep 18 20:22:08.318160 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:08.318087    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5\": container with ID starting with 9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5 not found: ID does not exist" containerID="9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"
Sep 18 20:22:08.318160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.318130    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5"} err="failed to get container status \"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5\": rpc error: code = NotFound desc = could not find container \"9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5\": container with ID starting with 9e06d995592e8a4b4ca9ef46624dfe1fc4abe3985231cf4909f6d7faff8c4ba5 not found: ID does not exist"
Sep 18 20:22:08.328869 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.328850    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:22:08.333894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:08.333873    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-802c6c2a-589c57dbcd-szf87"]
Sep 18 20:22:10.082024 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:10.081984    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" path="/var/lib/kubelet/pods/5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1/volumes"
Sep 18 20:22:13.181069 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.180973    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:22:13.181528 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.181230    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" podUID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" containerName="payload-processing" containerID="cri-o://f52f260e7a5ab1a46c468d45807a5188efa3e97176de34972085d73abbee993e" gracePeriod=30
Sep 18 20:22:13.186003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.185972    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:22:13.186874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.186191    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" podUID="467b902a-a909-4e80-90e8-fc211b517ced" containerName="payload-pre-processing" containerID="cri-o://5713188acf52d50ca7b382efee4b252a5345556aa58165794bb35179d1380f44" gracePeriod=30
Sep 18 20:22:13.353165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.352712    2566 generic.go:356] "Generic (PLEG): container finished" podID="467b902a-a909-4e80-90e8-fc211b517ced" containerID="5713188acf52d50ca7b382efee4b252a5345556aa58165794bb35179d1380f44" exitCode=0
Sep 18 20:22:13.353165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.352899    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" event={"ID":"467b902a-a909-4e80-90e8-fc211b517ced","Type":"ContainerDied","Data":"5713188acf52d50ca7b382efee4b252a5345556aa58165794bb35179d1380f44"}
Sep 18 20:22:13.353806 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.353743    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"]
Sep 18 20:22:13.356131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.356045    2566 generic.go:356] "Generic (PLEG): container finished" podID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" containerID="f52f260e7a5ab1a46c468d45807a5188efa3e97176de34972085d73abbee993e" exitCode=0
Sep 18 20:22:13.356131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.356110    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" event={"ID":"e8a7e6ff-7d07-45a8-b875-11bfb5252d74","Type":"ContainerDied","Data":"f52f260e7a5ab1a46c468d45807a5188efa3e97176de34972085d73abbee993e"}
Sep 18 20:22:13.358544 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.358519    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-a-e2dea3-24x2v"]
Sep 18 20:22:13.483175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.483150    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:22:13.514362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.514336    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:22:13.608777 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.608748    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nkccp\" (UniqueName: \"kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp\") pod \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") "
Sep 18 20:22:13.608924 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.608909    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume\") pod \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\" (UID: \"e8a7e6ff-7d07-45a8-b875-11bfb5252d74\") "
Sep 18 20:22:13.608990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.608960    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume\") pod \"467b902a-a909-4e80-90e8-fc211b517ced\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") "
Sep 18 20:22:13.608990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.608980    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b2g64\" (UniqueName: \"kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64\") pod \"467b902a-a909-4e80-90e8-fc211b517ced\" (UID: \"467b902a-a909-4e80-90e8-fc211b517ced\") "
Sep 18 20:22:13.609259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.609236    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e8a7e6ff-7d07-45a8-b875-11bfb5252d74" (UID: "e8a7e6ff-7d07-45a8-b875-11bfb5252d74"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:13.609353 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.609322    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "467b902a-a909-4e80-90e8-fc211b517ced" (UID: "467b902a-a909-4e80-90e8-fc211b517ced"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:13.609353 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.609333    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:13.610929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.610904    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp" (OuterVolumeSpecName: "kube-api-access-nkccp") pod "e8a7e6ff-7d07-45a8-b875-11bfb5252d74" (UID: "e8a7e6ff-7d07-45a8-b875-11bfb5252d74"). InnerVolumeSpecName "kube-api-access-nkccp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:13.611020 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.610951    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64" (OuterVolumeSpecName: "kube-api-access-b2g64") pod "467b902a-a909-4e80-90e8-fc211b517ced" (UID: "467b902a-a909-4e80-90e8-fc211b517ced"). InnerVolumeSpecName "kube-api-access-b2g64". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:13.710599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.710573    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/467b902a-a909-4e80-90e8-fc211b517ced-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:13.710599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.710596    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b2g64\" (UniqueName: \"kubernetes.io/projected/467b902a-a909-4e80-90e8-fc211b517ced-kube-api-access-b2g64\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:13.710599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:13.710605    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nkccp\" (UniqueName: \"kubernetes.io/projected/e8a7e6ff-7d07-45a8-b875-11bfb5252d74-kube-api-access-nkccp\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:14.081975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.081944    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37218b7b-fed4-444d-a743-b4d7c66b33f0" path="/var/lib/kubelet/pods/37218b7b-fed4-444d-a743-b4d7c66b33f0/volumes"
Sep 18 20:22:14.362927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.362859    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"
Sep 18 20:22:14.362927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.362866    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29" event={"ID":"e8a7e6ff-7d07-45a8-b875-11bfb5252d74","Type":"ContainerDied","Data":"426b1681ff124bb8e7a2d9c8e6a40ed60f5aad0a96c43d867f5adeb6fb69aaa9"}
Sep 18 20:22:14.362927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.362922    2566 scope.go:117] "RemoveContainer" containerID="f52f260e7a5ab1a46c468d45807a5188efa3e97176de34972085d73abbee993e"
Sep 18 20:22:14.364450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.364435    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"
Sep 18 20:22:14.364450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.364440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk" event={"ID":"467b902a-a909-4e80-90e8-fc211b517ced","Type":"ContainerDied","Data":"94de9d53bf512d237f2f36c30f804c9c861dfbaf10261ea7e82a70045e405fc8"}
Sep 18 20:22:14.372472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.372455    2566 scope.go:117] "RemoveContainer" containerID="5713188acf52d50ca7b382efee4b252a5345556aa58165794bb35179d1380f44"
Sep 18 20:22:14.381334 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.381315    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:22:14.385224 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.385207    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-a-e2dea3-77b58d8ffd-x5vkk"]
Sep 18 20:22:14.396095 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.396075    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:22:14.401383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.401365    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-a-e2dea3-9998d5fcc-kgq29"]
Sep 18 20:22:14.818566 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.818537    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:22:14.818749 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:14.818725    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy" containerID="cri-o://c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a" gracePeriod=30
Sep 18 20:22:16.082126 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:16.082096    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="467b902a-a909-4e80-90e8-fc211b517ced" path="/var/lib/kubelet/pods/467b902a-a909-4e80-90e8-fc211b517ced/volumes"
Sep 18 20:22:16.082480 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:16.082454    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" path="/var/lib/kubelet/pods/e8a7e6ff-7d07-45a8-b875-11bfb5252d74/volumes"
Sep 18 20:22:17.407375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.407315    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" podUID="ddc9f839-7a9e-44e5-8721-c649469805f1" containerName="main" containerID="cri-o://4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad" gracePeriod=45
Sep 18 20:22:17.667848 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.667791    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:22:17.755519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755491    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755570    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l7b6d\" (UniqueName: \"kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755598    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755641    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755729    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir\") pod \"ddc9f839-7a9e-44e5-8721-c649469805f1\" (UID: \"ddc9f839-7a9e-44e5-8721-c649469805f1\") "
Sep 18 20:22:17.755927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755872    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home" (OuterVolumeSpecName: "home") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:17.755983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.755955    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache" (OuterVolumeSpecName: "model-cache") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:17.756164 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.756129    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:17.756285 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.756167    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-home\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:17.756285 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.756190    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-model-cache\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:17.757816 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.757791    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm" (OuterVolumeSpecName: "dshm") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:17.757919 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.757899    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d" (OuterVolumeSpecName: "kube-api-access-l7b6d") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "kube-api-access-l7b6d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:17.757969 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.757949    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "ddc9f839-7a9e-44e5-8721-c649469805f1" (UID: "ddc9f839-7a9e-44e5-8721-c649469805f1"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:22:17.857051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.857022    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ddc9f839-7a9e-44e5-8721-c649469805f1-tls-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:17.857189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.857045    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-dshm\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:17.857189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.857086    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l7b6d\" (UniqueName: \"kubernetes.io/projected/ddc9f839-7a9e-44e5-8721-c649469805f1-kube-api-access-l7b6d\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:17.857189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:17.857096    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ddc9f839-7a9e-44e5-8721-c649469805f1-tmp-dir\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:18.392351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.392268    2566 generic.go:356] "Generic (PLEG): container finished" podID="ddc9f839-7a9e-44e5-8721-c649469805f1" containerID="4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad" exitCode=0
Sep 18 20:22:18.392351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.392303    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" event={"ID":"ddc9f839-7a9e-44e5-8721-c649469805f1","Type":"ContainerDied","Data":"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"}
Sep 18 20:22:18.392351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.392320    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"
Sep 18 20:22:18.392351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.392343    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6" event={"ID":"ddc9f839-7a9e-44e5-8721-c649469805f1","Type":"ContainerDied","Data":"5279c927e3f4445a4cfb7a9ce26d2d4cd5cc006745b4163283855c58036bb21d"}
Sep 18 20:22:18.392629 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.392364    2566 scope.go:117] "RemoveContainer" containerID="4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"
Sep 18 20:22:18.404713 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.404695    2566 scope.go:117] "RemoveContainer" containerID="4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"
Sep 18 20:22:18.404981 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:18.404954    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad\": container with ID starting with 4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad not found: ID does not exist" containerID="4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"
Sep 18 20:22:18.405036 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.404980    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad"} err="failed to get container status \"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad\": rpc error: code = NotFound desc = could not find container \"4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad\": container with ID starting with 4e93750060774a5f7058c28f889aa122988b9a7d7956611b92abd72ab8f29cad not found: ID does not exist"
Sep 18 20:22:18.411728 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.411706    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:22:18.415207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:18.415186    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-b-a85f8b/test-model-a85f8b-kserve-66d77b8bd5-b74h6"]
Sep 18 20:22:19.834816 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:19.834790    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:22:19.835210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:19.834999    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy" containerID="cri-o://6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747" gracePeriod=30
Sep 18 20:22:20.081537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.081474    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ddc9f839-7a9e-44e5-8721-c649469805f1" path="/var/lib/kubelet/pods/ddc9f839-7a9e-44e5-8721-c649469805f1/volumes"
Sep 18 20:22:20.095420 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.095397    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:20.175803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.175772    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.175943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.175820    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.175943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.175882    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.175943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.175905    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vrjpc\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.175959    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176032    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176004    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"]
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176028    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176074    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176138    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176157    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.176459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176219    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert\") pod \"16d9506b-f54f-45f1-952e-b20e424cd113\" (UID: \"16d9506b-f54f-45f1-952e-b20e424cd113\") "
Sep 18 20:22:20.176459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176403    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data" (OuterVolumeSpecName: "istio-data") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.176636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176613    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.176734 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176716    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176738    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176748    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176754    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176776    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.176796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176791    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176805    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176758    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176838    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37218b7b-fed4-444d-a743-b4d7c66b33f0" containerName="revoke-keys"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176851    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="37218b7b-fed4-444d-a743-b4d7c66b33f0" containerName="revoke-keys"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176874    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ddc9f839-7a9e-44e5-8721-c649469805f1" containerName="main"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176883    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ddc9f839-7a9e-44e5-8721-c649469805f1" containerName="main"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176904    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" containerName="payload-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176915    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" containerName="payload-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176931    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="467b902a-a909-4e80-90e8-fc211b517ced" containerName="payload-pre-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176940    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="467b902a-a909-4e80-90e8-fc211b517ced" containerName="payload-pre-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176955    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" containerName="payload-pre-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.176964    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" containerName="payload-pre-processing"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177130    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.177136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177146    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="37218b7b-fed4-444d-a743-b4d7c66b33f0" containerName="revoke-keys"
Sep 18 20:22:20.177611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177158    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="467b902a-a909-4e80-90e8-fc211b517ced" containerName="payload-pre-processing"
Sep 18 20:22:20.177611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177168    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8a7e6ff-7d07-45a8-b875-11bfb5252d74" containerName="payload-processing"
Sep 18 20:22:20.177611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177179    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ddc9f839-7a9e-44e5-8721-c649469805f1" containerName="main"
Sep 18 20:22:20.177611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177196    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" containerName="istio-proxy"
Sep 18 20:22:20.177611 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.177204    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="5abcbfe7-b0b7-4de3-b0e0-90dc6a716ff1" containerName="payload-pre-processing"
Sep 18 20:22:20.178593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.178565    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:20.178724 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.178708    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:22:20.178899 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.178877    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc" (OuterVolumeSpecName: "kube-api-access-vrjpc") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "kube-api-access-vrjpc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:20.178983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.178910    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token" (OuterVolumeSpecName: "istio-token") pod "16d9506b-f54f-45f1-952e-b20e424cd113" (UID: "16d9506b-f54f-45f1-952e-b20e424cd113"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:20.185556 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.185397    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:20.187607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.187518    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"]
Sep 18 20:22:20.188210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.188189    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:22:20.189377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.189357    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:22:20.278258 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278235    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt\" (UID: \"53ad2eca-ed5b-4d60-a45a-26217606abfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:20.278357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278303    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.278357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278314    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vrjpc\" (UniqueName: \"kubernetes.io/projected/16d9506b-f54f-45f1-952e-b20e424cd113-kube-api-access-vrjpc\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.278357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278322    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/16d9506b-f54f-45f1-952e-b20e424cd113-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.278357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278331    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/16d9506b-f54f-45f1-952e-b20e424cd113-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.278357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.278339    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/16d9506b-f54f-45f1-952e-b20e424cd113-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:20.379202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.379152    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt\" (UID: \"53ad2eca-ed5b-4d60-a45a-26217606abfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:20.379730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.379707    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt\" (UID: \"53ad2eca-ed5b-4d60-a45a-26217606abfa\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:20.406226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.406165    2566 generic.go:356] "Generic (PLEG): container finished" podID="16d9506b-f54f-45f1-952e-b20e424cd113" containerID="c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a" exitCode=0
Sep 18 20:22:20.406226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.406216    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"
Sep 18 20:22:20.406377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.406253    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerDied","Data":"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"}
Sep 18 20:22:20.406377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.406319    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8" event={"ID":"16d9506b-f54f-45f1-952e-b20e424cd113","Type":"ContainerDied","Data":"32bd27100f268a06d97e022363266b89422a51fa97b01af3571223215c8225d8"}
Sep 18 20:22:20.406377 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.406353    2566 scope.go:117] "RemoveContainer" containerID="c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"
Sep 18 20:22:20.415753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.415739    2566 scope.go:117] "RemoveContainer" containerID="8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"
Sep 18 20:22:20.423778 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.423756    2566 scope.go:117] "RemoveContainer" containerID="c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"
Sep 18 20:22:20.424037 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:20.424018    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a\": container with ID starting with c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a not found: ID does not exist" containerID="c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"
Sep 18 20:22:20.424108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.424045    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a"} err="failed to get container status \"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a\": rpc error: code = NotFound desc = could not find container \"c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a\": container with ID starting with c742c29ed18b2919791dd39d5e0499670c00d4aef3cfa24014795af8fde53d5a not found: ID does not exist"
Sep 18 20:22:20.424108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.424074    2566 scope.go:117] "RemoveContainer" containerID="8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"
Sep 18 20:22:20.424326 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:20.424297    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf\": container with ID starting with 8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf not found: ID does not exist" containerID="8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"
Sep 18 20:22:20.424360 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.424331    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf"} err="failed to get container status \"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf\": rpc error: code = NotFound desc = could not find container \"8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf\": container with ID starting with 8e8064050160a0e7473bf0a2c97b391dfe717d121a18feebf51e61c79fb8feaf not found: ID does not exist"
Sep 18 20:22:20.427760 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.427739    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:22:20.431652 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.431632    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-802c6c2a-openshift-default-5744bb75b8-6hxc8"]
Sep 18 20:22:20.506300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.506279    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:20.780013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:20.779988    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"]
Sep 18 20:22:20.781437 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:20.781406    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53ad2eca_ed5b_4d60_a45a_26217606abfa.slice/crio-4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48 WatchSource:0}: Error finding container 4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48: Status 404 returned error can't find the container with id 4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48
Sep 18 20:22:21.413967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:21.413880    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt" event={"ID":"53ad2eca-ed5b-4d60-a45a-26217606abfa","Type":"ContainerStarted","Data":"79c26ec5f6b6cbcf96e5707c0cd6b9f3516789aefbd869e64227cad79d0be435"}
Sep 18 20:22:21.413967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:21.413958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt" event={"ID":"53ad2eca-ed5b-4d60-a45a-26217606abfa","Type":"ContainerStarted","Data":"4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48"}
Sep 18 20:22:21.431639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:21.431597    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt" podStartSLOduration=1.431581373 podStartE2EDuration="1.431581373s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:21.430280198 +0000 UTC m=+1285.559582150" watchObservedRunningTime="2026-09-18 20:22:21.431581373 +0000 UTC m=+1285.560883326"
Sep 18 20:22:22.081105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:22.081047    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="16d9506b-f54f-45f1-952e-b20e424cd113" path="/var/lib/kubelet/pods/16d9506b-f54f-45f1-952e-b20e424cd113/volumes"
Sep 18 20:22:22.423596 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:22.423508    2566 generic.go:356] "Generic (PLEG): container finished" podID="53ad2eca-ed5b-4d60-a45a-26217606abfa" containerID="79c26ec5f6b6cbcf96e5707c0cd6b9f3516789aefbd869e64227cad79d0be435" exitCode=0
Sep 18 20:22:22.423927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:22.423606    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt" event={"ID":"53ad2eca-ed5b-4d60-a45a-26217606abfa","Type":"ContainerDied","Data":"79c26ec5f6b6cbcf96e5707c0cd6b9f3516789aefbd869e64227cad79d0be435"}
Sep 18 20:22:23.574889 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:23.574863    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:23.711651 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:23.711625    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca\") pod \"53ad2eca-ed5b-4d60-a45a-26217606abfa\" (UID: \"53ad2eca-ed5b-4d60-a45a-26217606abfa\") "
Sep 18 20:22:23.711940 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:23.711920    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "53ad2eca-ed5b-4d60-a45a-26217606abfa" (UID: "53ad2eca-ed5b-4d60-a45a-26217606abfa"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:23.812401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:23.812373    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/53ad2eca-ed5b-4d60-a45a-26217606abfa-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:24.436352 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:24.436286    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"
Sep 18 20:22:24.436485 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:24.436284    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt" event={"ID":"53ad2eca-ed5b-4d60-a45a-26217606abfa","Type":"ContainerDied","Data":"4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48"}
Sep 18 20:22:24.436485 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:24.436414    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="4d71e2c5b1c270dd2c54532ab7f0e7a76df25984441ad1b2f0afb0392198af48"
Sep 18 20:22:25.080231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.080209    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:22:25.224555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224532    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9tbks\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224578    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224603    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224622    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224636    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224652    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.224712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224682    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.225034 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224716    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.225034 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.224760    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo\") pod \"24891591-0b52-493b-8eb6-f0db69e999f0\" (UID: \"24891591-0b52-493b-8eb6-f0db69e999f0\") "
Sep 18 20:22:25.225034 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225002    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:25.225222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225083    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:25.225222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225140    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:25.225330 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225279    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data" (OuterVolumeSpecName: "istio-data") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:25.225418 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225393    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:25.225492 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225474    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.225555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225498    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.225555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.225513    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.226985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.226935    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks" (OuterVolumeSpecName: "kube-api-access-9tbks") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "kube-api-access-9tbks". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:25.226985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.226966    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:22:25.227213 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.227026    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:22:25.227213 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.227050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token" (OuterVolumeSpecName: "istio-token") pod "24891591-0b52-493b-8eb6-f0db69e999f0" (UID: "24891591-0b52-493b-8eb6-f0db69e999f0"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:25.326299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326272    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9tbks\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-kube-api-access-9tbks\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.326299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326296    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.326440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326305    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/24891591-0b52-493b-8eb6-f0db69e999f0-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.326440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326313    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/24891591-0b52-493b-8eb6-f0db69e999f0-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.326440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326321    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/24891591-0b52-493b-8eb6-f0db69e999f0-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.326440 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.326328    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/24891591-0b52-493b-8eb6-f0db69e999f0-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:25.446135 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.446104    2566 generic.go:356] "Generic (PLEG): container finished" podID="24891591-0b52-493b-8eb6-f0db69e999f0" containerID="6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747" exitCode=0
Sep 18 20:22:25.446239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.446158    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"
Sep 18 20:22:25.446239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.446159    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerDied","Data":"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"}
Sep 18 20:22:25.446239 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.446231    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6" event={"ID":"24891591-0b52-493b-8eb6-f0db69e999f0","Type":"ContainerDied","Data":"8f1df4953d36eb1b72d67c098a253396159cfd9dd90305668ce0eeb66074a0fd"}
Sep 18 20:22:25.446376 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.446261    2566 scope.go:117] "RemoveContainer" containerID="6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"
Sep 18 20:22:25.456799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.456782    2566 scope.go:117] "RemoveContainer" containerID="bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"
Sep 18 20:22:25.465282 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.465267    2566 scope.go:117] "RemoveContainer" containerID="6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"
Sep 18 20:22:25.465571 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:25.465552    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747\": container with ID starting with 6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747 not found: ID does not exist" containerID="6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"
Sep 18 20:22:25.465643 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.465578    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747"} err="failed to get container status \"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747\": rpc error: code = NotFound desc = could not find container \"6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747\": container with ID starting with 6b19a9f5117785f3065087874fed16487ebfabb322f285826421344739a2b747 not found: ID does not exist"
Sep 18 20:22:25.465643 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.465594    2566 scope.go:117] "RemoveContainer" containerID="bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"
Sep 18 20:22:25.465833 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:22:25.465805    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616\": container with ID starting with bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616 not found: ID does not exist" containerID="bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"
Sep 18 20:22:25.465880 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.465828    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616"} err="failed to get container status \"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616\": rpc error: code = NotFound desc = could not find container \"bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616\": container with ID starting with bb80f09dfaf7d7c987f65b1ee2c5af590aefeb26870c0e9af676f39ea5c08616 not found: ID does not exist"
Sep 18 20:22:25.469888 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.469868    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:22:25.478873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:25.478797    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-inf-a-e2dea3-openshift-default-5445bf7c74-ps2q6"]
Sep 18 20:22:26.081280 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:26.081248    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" path="/var/lib/kubelet/pods/24891591-0b52-493b-8eb6-f0db69e999f0/volumes"
Sep 18 20:22:28.432643 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.432612    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:22:28.433097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433082    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.433097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433098    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.433192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433114    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.433192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433123    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.433192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433131    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="53ad2eca-ed5b-4d60-a45a-26217606abfa" containerName="revoke-keys"
Sep 18 20:22:28.433192 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433136    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="53ad2eca-ed5b-4d60-a45a-26217606abfa" containerName="revoke-keys"
Sep 18 20:22:28.433337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433212    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.433337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433220    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="53ad2eca-ed5b-4d60-a45a-26217606abfa" containerName="revoke-keys"
Sep 18 20:22:28.433337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.433227    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="24891591-0b52-493b-8eb6-f0db69e999f0" containerName="istio-proxy"
Sep 18 20:22:28.437927 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.437910    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.440446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.440423    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w1-35f893-openshift-default-dockercfg-wjm4g\""
Sep 18 20:22:28.450095 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.450049    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:22:28.555854 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.555822    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.555867    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.555908    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.555951    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.556005    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.556122    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.556169    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.556198    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.556291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.556237    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p4sbv\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657550 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657518    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657562    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657585    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657623    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657643    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657658    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657687    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p4sbv\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.657772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657786    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.657935    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.658038    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.658133    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.658287    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.658659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.658643    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.659991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.659956    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.660243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.660225    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.665482 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.665457    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.665589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.665520    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p4sbv\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv\") pod \"e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") " pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:28.750168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:28.750140    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:29.027632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.027602    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:22:29.029140 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:29.029115    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d97ba05_1dcb_4995_a4c0_0b8c8981c062.slice/crio-68d5107338becf51191e761c64d7c0bb622e88dafd0c350d2bf742497e06cb21 WatchSource:0}: Error finding container 68d5107338becf51191e761c64d7c0bb622e88dafd0c350d2bf742497e06cb21: Status 404 returned error can't find the container with id 68d5107338becf51191e761c64d7c0bb622e88dafd0c350d2bf742497e06cb21
Sep 18 20:22:29.034756 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.034727    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:29.034865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.034801    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:29.034865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.034846    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:29.472874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.472836    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerStarted","Data":"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"}
Sep 18 20:22:29.473314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.472906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerStarted","Data":"68d5107338becf51191e761c64d7c0bb622e88dafd0c350d2bf742497e06cb21"}
Sep 18 20:22:29.492931 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.492886    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podStartSLOduration=1.492871351 podStartE2EDuration="1.492871351s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:29.491983537 +0000 UTC m=+1293.621285501" watchObservedRunningTime="2026-09-18 20:22:29.492871351 +0000 UTC m=+1293.622173301"
Sep 18 20:22:29.750358 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:29.750264    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:30.181607 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.181528    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:22:30.185554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.185534    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.188697 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.188675    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"openshift-service-ca.crt\""
Sep 18 20:22:30.188823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.188745    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"default-dockercfg-phwfg\""
Sep 18 20:22:30.188823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.188780    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"distinct-2-w0-116778-kserve-self-signed-certs\""
Sep 18 20:22:30.188823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.188804    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-116778\"/\"kube-root-ca.crt\""
Sep 18 20:22:30.195753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.195733    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.273672    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.273782    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.273849    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.273876    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.273949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.274519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.274097    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lv9m\" (UniqueName: \"kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375170    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375209    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375240    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375272    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lv9m\" (UniqueName: \"kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375603 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375584    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375658 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375614    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.375697 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.375687    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.377483 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.377465    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.377744 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.377727    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.383111 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.383090    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lv9m\" (UniqueName: \"kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m\") pod \"distinct-2-w0-116778-kserve-74969f55f8-shrxp\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") " pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.496222 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.496188    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:30.756924 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.756850    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:30.771953 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:30.771930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:22:30.773823 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:30.773798    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8743f4d8_9cef_450e_a51c_53813857640a.slice/crio-afa72153b2b4c2ed6611da2dc12d8f0b94dace2c1cc0c38daf5e2f8042b55c3a WatchSource:0}: Error finding container afa72153b2b4c2ed6611da2dc12d8f0b94dace2c1cc0c38daf5e2f8042b55c3a: Status 404 returned error can't find the container with id afa72153b2b4c2ed6611da2dc12d8f0b94dace2c1cc0c38daf5e2f8042b55c3a
Sep 18 20:22:31.486930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:31.486893    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" event={"ID":"8743f4d8-9cef-450e-a51c-53813857640a","Type":"ContainerStarted","Data":"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"}
Sep 18 20:22:31.487099 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:31.486940    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" event={"ID":"8743f4d8-9cef-450e-a51c-53813857640a","Type":"ContainerStarted","Data":"afa72153b2b4c2ed6611da2dc12d8f0b94dace2c1cc0c38daf5e2f8042b55c3a"}
Sep 18 20:22:31.487268 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:31.487247    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:31.488042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:31.488022    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:31.508937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:31.508899    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" podStartSLOduration=1.508884998 podStartE2EDuration="1.508884998s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:31.506678845 +0000 UTC m=+1295.635980795" watchObservedRunningTime="2026-09-18 20:22:31.508884998 +0000 UTC m=+1295.638186983"
Sep 18 20:22:33.275836 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.275804    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:22:33.276314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.276082    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" podUID="587cadfb-dd4d-4610-b2d4-93316abd4d34" containerName="payload-processing" containerID="cri-o://17f6d52267dec896ce138ef63b9eba50466d4b90567d629a2eecf2ec46ad5e0e" gracePeriod=30
Sep 18 20:22:33.283951 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.283926    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:22:33.285232 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.284990    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" podUID="1bb0f727-3591-4cfd-8f20-7eff6df56851" containerName="payload-pre-processing" containerID="cri-o://91064f9b7703dfc98bec945daea6aa4c980dbac6871e913cd665b67d96ba1301" gracePeriod=30
Sep 18 20:22:33.452630 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.452213    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"]
Sep 18 20:22:33.455085 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.454783    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-inf-b-a85f8b-fgnbt"]
Sep 18 20:22:33.506094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.506026    2566 generic.go:356] "Generic (PLEG): container finished" podID="1bb0f727-3591-4cfd-8f20-7eff6df56851" containerID="91064f9b7703dfc98bec945daea6aa4c980dbac6871e913cd665b67d96ba1301" exitCode=0
Sep 18 20:22:33.506242 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.506173    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" event={"ID":"1bb0f727-3591-4cfd-8f20-7eff6df56851","Type":"ContainerDied","Data":"91064f9b7703dfc98bec945daea6aa4c980dbac6871e913cd665b67d96ba1301"}
Sep 18 20:22:33.508484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.508380    2566 generic.go:356] "Generic (PLEG): container finished" podID="587cadfb-dd4d-4610-b2d4-93316abd4d34" containerID="17f6d52267dec896ce138ef63b9eba50466d4b90567d629a2eecf2ec46ad5e0e" exitCode=0
Sep 18 20:22:33.508484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.508438    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" event={"ID":"587cadfb-dd4d-4610-b2d4-93316abd4d34","Type":"ContainerDied","Data":"17f6d52267dec896ce138ef63b9eba50466d4b90567d629a2eecf2ec46ad5e0e"}
Sep 18 20:22:33.574203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.574178    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:22:33.577219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.577177    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:22:33.717115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717087    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume\") pod \"587cadfb-dd4d-4610-b2d4-93316abd4d34\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") "
Sep 18 20:22:33.717303 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717190    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b9cnq\" (UniqueName: \"kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq\") pod \"1bb0f727-3591-4cfd-8f20-7eff6df56851\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") "
Sep 18 20:22:33.717303 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717243    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-thxwd\" (UniqueName: \"kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd\") pod \"587cadfb-dd4d-4610-b2d4-93316abd4d34\" (UID: \"587cadfb-dd4d-4610-b2d4-93316abd4d34\") "
Sep 18 20:22:33.717303 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717270    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume\") pod \"1bb0f727-3591-4cfd-8f20-7eff6df56851\" (UID: \"1bb0f727-3591-4cfd-8f20-7eff6df56851\") "
Sep 18 20:22:33.717616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717509    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "587cadfb-dd4d-4610-b2d4-93316abd4d34" (UID: "587cadfb-dd4d-4610-b2d4-93316abd4d34"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:33.717737 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.717662    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "1bb0f727-3591-4cfd-8f20-7eff6df56851" (UID: "1bb0f727-3591-4cfd-8f20-7eff6df56851"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:22:33.719223 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.719205    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd" (OuterVolumeSpecName: "kube-api-access-thxwd") pod "587cadfb-dd4d-4610-b2d4-93316abd4d34" (UID: "587cadfb-dd4d-4610-b2d4-93316abd4d34"). InnerVolumeSpecName "kube-api-access-thxwd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:33.719307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.719234    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq" (OuterVolumeSpecName: "kube-api-access-b9cnq") pod "1bb0f727-3591-4cfd-8f20-7eff6df56851" (UID: "1bb0f727-3591-4cfd-8f20-7eff6df56851"). InnerVolumeSpecName "kube-api-access-b9cnq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:22:33.818247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.818171    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b9cnq\" (UniqueName: \"kubernetes.io/projected/1bb0f727-3591-4cfd-8f20-7eff6df56851-kube-api-access-b9cnq\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.818247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.818196    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-thxwd\" (UniqueName: \"kubernetes.io/projected/587cadfb-dd4d-4610-b2d4-93316abd4d34-kube-api-access-thxwd\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.818247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.818207    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/1bb0f727-3591-4cfd-8f20-7eff6df56851-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:33.818247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:33.818215    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/587cadfb-dd4d-4610-b2d4-93316abd4d34-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:22:34.081776 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.081698    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="53ad2eca-ed5b-4d60-a45a-26217606abfa" path="/var/lib/kubelet/pods/53ad2eca-ed5b-4d60-a45a-26217606abfa/volumes"
Sep 18 20:22:34.197156 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197120    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:22:34.197667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197643    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1bb0f727-3591-4cfd-8f20-7eff6df56851" containerName="payload-pre-processing"
Sep 18 20:22:34.197667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197668    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1bb0f727-3591-4cfd-8f20-7eff6df56851" containerName="payload-pre-processing"
Sep 18 20:22:34.197825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197693    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="587cadfb-dd4d-4610-b2d4-93316abd4d34" containerName="payload-processing"
Sep 18 20:22:34.197825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197704    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="587cadfb-dd4d-4610-b2d4-93316abd4d34" containerName="payload-processing"
Sep 18 20:22:34.197825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197815    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1bb0f727-3591-4cfd-8f20-7eff6df56851" containerName="payload-pre-processing"
Sep 18 20:22:34.197825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.197824    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="587cadfb-dd4d-4610-b2d4-93316abd4d34" containerName="payload-processing"
Sep 18 20:22:34.209027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.209003    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.211551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.211531    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w1-35f893-dockercfg-s5hcm\""
Sep 18 20:22:34.211662 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.211572    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w1-35f893\""
Sep 18 20:22:34.212112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.212091    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:22:34.220875 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.220854    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f74pq\" (UniqueName: \"kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.220958 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.220914    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.244511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.244486    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:22:34.247936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.247918    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.259372 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.259350    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:22:34.324083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.322119    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.325075 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.325032    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.326009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.325283    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xqfp2\" (UniqueName: \"kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.326009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.325477    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f74pq\" (UniqueName: \"kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.326009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.325352    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.341605 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.339688    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f74pq\" (UniqueName: \"kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq\") pod \"payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.427266 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.427238    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.427390 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.427315    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xqfp2\" (UniqueName: \"kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.427905 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.427885    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.434965 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.434937    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xqfp2\" (UniqueName: \"kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2\") pod \"payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.515351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.515330    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"
Sep 18 20:22:34.515490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.515329    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg" event={"ID":"1bb0f727-3591-4cfd-8f20-7eff6df56851","Type":"ContainerDied","Data":"11d9092c0e5e3f305cde0a1179b186f580aedc4326d6d65d63db475e972b17b6"}
Sep 18 20:22:34.515490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.515459    2566 scope.go:117] "RemoveContainer" containerID="91064f9b7703dfc98bec945daea6aa4c980dbac6871e913cd665b67d96ba1301"
Sep 18 20:22:34.516900 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.516883    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"
Sep 18 20:22:34.516996 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.516906    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl" event={"ID":"587cadfb-dd4d-4610-b2d4-93316abd4d34","Type":"ContainerDied","Data":"d2119f6f300d99e9b0197741eeecf280257e89842c55ea417df2c1ccd60d356d"}
Sep 18 20:22:34.519955 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.519934    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:34.524988 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.524964    2566 scope.go:117] "RemoveContainer" containerID="17f6d52267dec896ce138ef63b9eba50466d4b90567d629a2eecf2ec46ad5e0e"
Sep 18 20:22:34.543471 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.543440    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:22:34.545821 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.545799    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-inf-b-a85f8b-67b5868cff-rmlsl"]
Sep 18 20:22:34.554873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.554849    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:22:34.557936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.557914    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:34.563231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.563211    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-a85f8b-849b65786-6n7mg"]
Sep 18 20:22:34.811616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.811593    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:22:34.813345 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:34.813315    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod75971fd6_a93f_4958_ba8b_ea475b3cc36a.slice/crio-0295a81bd4f79ef15c02ab0631ab37bda3fac43271808e63df8061c88c446a24 WatchSource:0}: Error finding container 0295a81bd4f79ef15c02ab0631ab37bda3fac43271808e63df8061c88c446a24: Status 404 returned error can't find the container with id 0295a81bd4f79ef15c02ab0631ab37bda3fac43271808e63df8061c88c446a24
Sep 18 20:22:34.843209 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:34.843174    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:22:34.846341 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:34.846301    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4356f317_a2b8_47e4_a7ff_b1d8b953467a.slice/crio-e5fd3e4c06f17e3a24aa9ef9247f8b5bb0f033595a33c7bd7be90c88fe0e0682 WatchSource:0}: Error finding container e5fd3e4c06f17e3a24aa9ef9247f8b5bb0f033595a33c7bd7be90c88fe0e0682: Status 404 returned error can't find the container with id e5fd3e4c06f17e3a24aa9ef9247f8b5bb0f033595a33c7bd7be90c88fe0e0682
Sep 18 20:22:35.526371 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.526338    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" event={"ID":"75971fd6-a93f-4958-ba8b-ea475b3cc36a","Type":"ContainerStarted","Data":"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"}
Sep 18 20:22:35.526819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.526390    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" event={"ID":"75971fd6-a93f-4958-ba8b-ea475b3cc36a","Type":"ContainerStarted","Data":"0295a81bd4f79ef15c02ab0631ab37bda3fac43271808e63df8061c88c446a24"}
Sep 18 20:22:35.526819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.526446    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:35.528020 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.527994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" event={"ID":"4356f317-a2b8-47e4-a7ff-b1d8b953467a","Type":"ContainerStarted","Data":"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"}
Sep 18 20:22:35.528154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.528031    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" event={"ID":"4356f317-a2b8-47e4-a7ff-b1d8b953467a","Type":"ContainerStarted","Data":"e5fd3e4c06f17e3a24aa9ef9247f8b5bb0f033595a33c7bd7be90c88fe0e0682"}
Sep 18 20:22:35.528154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.528118    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:35.543827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.543779    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" podStartSLOduration=1.5437617829999999 podStartE2EDuration="1.543761783s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:35.540553581 +0000 UTC m=+1299.669855536" watchObservedRunningTime="2026-09-18 20:22:35.543761783 +0000 UTC m=+1299.673063735"
Sep 18 20:22:35.559002 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:35.558954    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" podStartSLOduration=1.5589378699999998 podStartE2EDuration="1.55893787s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:35.557515839 +0000 UTC m=+1299.686817816" watchObservedRunningTime="2026-09-18 20:22:35.55893787 +0000 UTC m=+1299.688239823"
Sep 18 20:22:36.080673 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:36.080642    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1bb0f727-3591-4cfd-8f20-7eff6df56851" path="/var/lib/kubelet/pods/1bb0f727-3591-4cfd-8f20-7eff6df56851/volumes"
Sep 18 20:22:36.081026 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:36.081012    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="587cadfb-dd4d-4610-b2d4-93316abd4d34" path="/var/lib/kubelet/pods/587cadfb-dd4d-4610-b2d4-93316abd4d34/volumes"
Sep 18 20:22:40.497157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.497124    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:40.497617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.497171    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:40.499295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.499262    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:22:40.504880 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.504862    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.507124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.507105    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-bab50d-openshift-default-dockercfg-6rklg\""
Sep 18 20:22:40.510505 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.510481    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:40.515562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.515540    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:22:40.582407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.582380    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:22:40.588521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588497    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.588716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588696    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.588862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588829    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.588862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588875    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.589092 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588949    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.589092 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.588990    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.589211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.589160    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fdjw\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.589267 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.589232    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.589319 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.589265    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.689805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.689774    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.689986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.689818    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.689986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.689939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5fdjw\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690007    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690039    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690122    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690150    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690178    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690189    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690211    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690544    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690670 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690628    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690749 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690684    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.690749 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.690740    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.692599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.692578    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.692690 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.692660    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.697786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.697764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.697921 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.697905    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fdjw\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw\") pod \"e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") " pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:40.818344 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:40.818280    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:41.094891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.094862    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:22:41.095935 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:41.095895    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode584c927_ba85_4533_ab33_2b496e361960.slice/crio-73fec062403c7907f185937c2509316d04d86ce4d2651c809f55dd95252a3d22 WatchSource:0}: Error finding container 73fec062403c7907f185937c2509316d04d86ce4d2651c809f55dd95252a3d22: Status 404 returned error can't find the container with id 73fec062403c7907f185937c2509316d04d86ce4d2651c809f55dd95252a3d22
Sep 18 20:22:41.098272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.098239    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:41.098365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.098329    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:41.098365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.098358    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:41.578163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.578127    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerStarted","Data":"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"}
Sep 18 20:22:41.578526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.578181    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerStarted","Data":"73fec062403c7907f185937c2509316d04d86ce4d2651c809f55dd95252a3d22"}
Sep 18 20:22:41.601094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.601014    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podStartSLOduration=1.600993811 podStartE2EDuration="1.600993811s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:41.59720246 +0000 UTC m=+1305.726504422" watchObservedRunningTime="2026-09-18 20:22:41.600993811 +0000 UTC m=+1305.730295768"
Sep 18 20:22:41.818492 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:41.818463    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:42.824327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:42.824298    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:43.589562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:43.589525    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:43.590395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:43.590379    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:22:46.536599 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.536571    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:22:46.536957 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.536641    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:22:46.765416 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.765384    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:22:46.769154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.769124    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.771754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.771727    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-b-w1-9a5190-openshift-default-dockercfg-dnqlj\""
Sep 18 20:22:46.783934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.783913    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:22:46.856074 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.855966    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856212 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856142    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856280    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856310    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856343    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856368    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856426    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vfrn\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.856554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.856496    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957662    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957715    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957730    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957849    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957891    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.957943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957922    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958156 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.957966    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vfrn\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958156 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958041    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958156 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958129    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958156 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958147    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958366 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958167    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958366 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958289    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958419    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.958769 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.958746    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.960583 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.960554    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.960786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.960769    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.966573 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.966554    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:22:46.975142 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.975118    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.975240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.975125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vfrn\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn\") pod \"e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") " pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:46.977385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.977371    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:46.980166 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.980135    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-bab50d\""
Sep 18 20:22:46.980386 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.980359    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-bab50d-dockercfg-454mx\""
Sep 18 20:22:46.982194 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:46.982172    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:22:47.059732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.059703    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k46g7\" (UniqueName: \"kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.059869 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.059851    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.082680 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.082637    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:47.160713 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.160679    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k46g7\" (UniqueName: \"kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.161084 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.160897    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.162410 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.162382    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.174554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.174521    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k46g7\" (UniqueName: \"kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7\") pod \"payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.303079 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.303028    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:47.403078 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.399930    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:22:47.426811 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:47.426780    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb978ea71_2c96_4928_a4fb_560cec27d317.slice/crio-1e19ccf3293c1897c0233a3079689a60bd7b9b34a1ee3534f38e08e8ffcdb42d WatchSource:0}: Error finding container 1e19ccf3293c1897c0233a3079689a60bd7b9b34a1ee3534f38e08e8ffcdb42d: Status 404 returned error can't find the container with id 1e19ccf3293c1897c0233a3079689a60bd7b9b34a1ee3534f38e08e8ffcdb42d
Sep 18 20:22:47.429227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.429136    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:47.429434 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.429385    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:47.429593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.429537    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:47.612544 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.610696    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:22:47.613762 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:22:47.613727    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea4c0892_03bf_4fd8_95fb_cf019eb8abfd.slice/crio-f2cd2e70b6ba9193cdd5e60bfc986acb9c65f16b2696eb6ca36a211ecc30eaa6 WatchSource:0}: Error finding container f2cd2e70b6ba9193cdd5e60bfc986acb9c65f16b2696eb6ca36a211ecc30eaa6: Status 404 returned error can't find the container with id f2cd2e70b6ba9193cdd5e60bfc986acb9c65f16b2696eb6ca36a211ecc30eaa6
Sep 18 20:22:47.621088 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.620968    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerStarted","Data":"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"}
Sep 18 20:22:47.621088 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:47.621029    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerStarted","Data":"1e19ccf3293c1897c0233a3079689a60bd7b9b34a1ee3534f38e08e8ffcdb42d"}
Sep 18 20:22:48.093643 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.093595    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:22:48.093950 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.093906    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:22:48.099639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.099609    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:48.629194 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.629151    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" event={"ID":"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd","Type":"ContainerStarted","Data":"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"}
Sep 18 20:22:48.629685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.629214    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" event={"ID":"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd","Type":"ContainerStarted","Data":"f2cd2e70b6ba9193cdd5e60bfc986acb9c65f16b2696eb6ca36a211ecc30eaa6"}
Sep 18 20:22:48.629685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.629379    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:48.646713 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.646653    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podStartSLOduration=2.646634217 podStartE2EDuration="2.646634217s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:47.646371707 +0000 UTC m=+1311.775673650" watchObservedRunningTime="2026-09-18 20:22:48.646634217 +0000 UTC m=+1312.775936169"
Sep 18 20:22:48.649890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:48.649839    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" podStartSLOduration=2.649822174 podStartE2EDuration="2.649822174s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:22:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:22:48.645089732 +0000 UTC m=+1312.774391681" watchObservedRunningTime="2026-09-18 20:22:48.649822174 +0000 UTC m=+1312.779124125"
Sep 18 20:22:49.087158 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:49.087131    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:49.636456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:49.636426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:49.637574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:49.637550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:22:56.695361 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.695276    2566 generic.go:356] "Generic (PLEG): container finished" podID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerID="1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd" exitCode=0
Sep 18 20:22:56.695778 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.695351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerDied","Data":"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"}
Sep 18 20:22:56.695778 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.695742    2566 scope.go:117] "RemoveContainer" containerID="1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"
Sep 18 20:22:56.696969 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.696946    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:56.697038 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.697011    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:56.697097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:56.697044    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:22:57.703957 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:57.703919    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerStarted","Data":"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"}
Sep 18 20:22:57.750997 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:57.750963    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:22:57.752540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:57.752498    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:22:57.752686 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:57.752571    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 20:22:58.761081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:58.760147    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:22:58.761081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:58.760216    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 20:22:59.638511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:59.638483    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:22:59.752079 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:59.751440    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:22:59.752079 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:22:59.751548    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 20:23:00.751647 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:00.751604    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:23:00.752165 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:00.751679    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 20:23:01.487152 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:01.487119    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:23:01.751551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:01.751451    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused" start-of-body=
Sep 18 20:23:01.751551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:01.751523    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.71:15021/healthz/ready\": dial tcp 10.132.0.71:15021: connect: connection refused"
Sep 18 20:23:02.756207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:02.756170    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:23:02.757131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:02.757109    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:23:14.124422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.124376    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:23:14.129731 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.129714    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.132233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.132214    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-b-w2-35558b-dockercfg-db87d\""
Sep 18 20:23:14.132474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.132457    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-b-w2-35558b\""
Sep 18 20:23:14.138706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.138684    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:23:14.207216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.205792    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tthnr\" (UniqueName: \"kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.207216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.205892    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.306889 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.306851    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tthnr\" (UniqueName: \"kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.307228 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.307208    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.307863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.307844    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.315827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.315580    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tthnr\" (UniqueName: \"kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr\") pod \"payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.501743 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.501704    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:14.837170 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:14.837130    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:23:14.848082 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:23:14.839019    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod094df03c_f1a1_4a3b_9032_3eeb138f1022.slice/crio-95ebe8df54bbfc13c9b2f303ca0a12aabfcc25c4b7760174d9d8f3e1c2af4f7e WatchSource:0}: Error finding container 95ebe8df54bbfc13c9b2f303ca0a12aabfcc25c4b7760174d9d8f3e1c2af4f7e: Status 404 returned error can't find the container with id 95ebe8df54bbfc13c9b2f303ca0a12aabfcc25c4b7760174d9d8f3e1c2af4f7e
Sep 18 20:23:15.823888 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:15.823851    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" event={"ID":"094df03c-f1a1-4a3b-9032-3eeb138f1022","Type":"ContainerStarted","Data":"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"}
Sep 18 20:23:15.824295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:15.823916    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" event={"ID":"094df03c-f1a1-4a3b-9032-3eeb138f1022","Type":"ContainerStarted","Data":"95ebe8df54bbfc13c9b2f303ca0a12aabfcc25c4b7760174d9d8f3e1c2af4f7e"}
Sep 18 20:23:15.824295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:15.823944    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:15.847085 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:15.845153    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" podStartSLOduration=1.845132461 podStartE2EDuration="1.845132461s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:15.838658803 +0000 UTC m=+1339.967960758" watchObservedRunningTime="2026-09-18 20:23:15.845132461 +0000 UTC m=+1339.974434411"
Sep 18 20:23:18.847591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.847506    2566 generic.go:356] "Generic (PLEG): container finished" podID="e584c927-ba85-4533-ab33-2b496e361960" containerID="6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc" exitCode=0
Sep 18 20:23:18.847591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.847553    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerDied","Data":"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"}
Sep 18 20:23:18.848134 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.848116    2566 scope.go:117] "RemoveContainer" containerID="6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"
Sep 18 20:23:18.849499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.849461    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:18.849615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.849560    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:18.849615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:18.849608    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:19.813444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.812919    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": read tcp 10.132.0.2:44092->10.132.0.76:15021: read: connection reset by peer" start-of-body=
Sep 18 20:23:19.813444 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.812984    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": read tcp 10.132.0.2:44092->10.132.0.76:15021: read: connection reset by peer"
Sep 18 20:23:19.865144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.865108    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerStarted","Data":"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"}
Sep 18 20:23:19.868090 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.867701    2566 generic.go:356] "Generic (PLEG): container finished" podID="b978ea71-2c96-4928-a4fb-560cec27d317" containerID="babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472" exitCode=0
Sep 18 20:23:19.868090 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.867849    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerDied","Data":"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"}
Sep 18 20:23:19.868374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.868349    2566 scope.go:117] "RemoveContainer" containerID="babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"
Sep 18 20:23:19.869847 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.869808    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:19.869945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.869903    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:19.870012 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:19.869946    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:20.819131 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:20.819092    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:23:20.819412 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:20.819393    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 18 20:23:20.819458 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:20.819436    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 18 20:23:20.891081 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:20.888891    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerStarted","Data":"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"}
Sep 18 20:23:21.084274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:21.084161    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:23:21.084453 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:21.084303    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:21.084453 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:21.084344    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:21.818654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:21.818621    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 18 20:23:21.818830 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:21.818673    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 18 20:23:22.083169 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:22.083073    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:22.083701 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:22.083671    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:22.819112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:22.819049    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 18 20:23:22.819274 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:22.819145    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 18 20:23:23.083692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.083598    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:23.084301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.084267    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:23.313202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.313164    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:23:23.316731 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.316706    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.319245 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.319225    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-10f4b77d-openshift-default-dockercfg-cvrkm\""
Sep 18 20:23:23.330369 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.330344    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:23:23.416790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.416673    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvksd\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.416971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.416824    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.416971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.416856    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.416997    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.417023    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417253 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.417135    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417253 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.417197    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417253 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.417225    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.417402 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.417253    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.518846 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.518809    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.518931    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.518972    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519177    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519229    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519260    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519360 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519342    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvksd\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519408    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519439    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519513    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519780 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519754    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519828    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.519881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519863    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.520003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.519924    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.521368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.521340    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.522577 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.522560    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.526520 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.526498    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.527335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.527315    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvksd\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd\") pod \"e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") " pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.628573 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.628541    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:23.819154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.818787    2566 patch_prober.go:28] interesting pod/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused" start-of-body=
Sep 18 20:23:23.819154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.818840    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.75:15021/healthz/ready\": dial tcp 10.132.0.75:15021: connect: connection refused"
Sep 18 20:23:23.953958 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.953925    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:23:23.995042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.993872    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:23.995042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.993983    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:23.995042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:23.994075    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:23:24.087103 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.085454    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:24.087103 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.085521    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:24.822987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.822959    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:23:24.823384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.823361    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:23:24.824117 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.824099    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:23:24.918237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.918202    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerStarted","Data":"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"}
Sep 18 20:23:24.918429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.918265    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerStarted","Data":"d0f9ce20ca483799d8c301929070540509f6be4eb63db59f1e5fe4eff5038e29"}
Sep 18 20:23:24.937031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:24.936979    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podStartSLOduration=1.936961341 podStartE2EDuration="1.936961341s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:24.934784274 +0000 UTC m=+1349.064086223" watchObservedRunningTime="2026-09-18 20:23:24.936961341 +0000 UTC m=+1349.066263289"
Sep 18 20:23:25.083789 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:25.083700    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:25.084045 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:25.084019    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:25.628745 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:25.628706    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:25.630167 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:25.630143    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:23:25.630290 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:25.630199    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:23:26.083936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.083904    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused" start-of-body=
Sep 18 20:23:26.084203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.083951    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.76:15021/healthz/ready\": dial tcp 10.132.0.76:15021: connect: connection refused"
Sep 18 20:23:26.633308 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.633278    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:26.831786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.831755    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:23:26.932180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.932101    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:26.933187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:26.933166    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:23:27.113004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:27.112973    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:23:27.113524 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:27.113504    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:23:27.114602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:27.114579    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:23:32.470223 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.470189    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:23:32.473720 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.473703    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.476041 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.476020    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-10f4b77d\""
Sep 18 20:23:32.476177 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.476022    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-10f4b77d-dockercfg-tmz4t\""
Sep 18 20:23:32.484823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.484799    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:23:32.630910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.630874    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.630910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.630911    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mrns\" (UniqueName: \"kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.731782 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.731702    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.731782 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.731742    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mrns\" (UniqueName: \"kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.732472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.732452    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.739797 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.739747    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mrns\" (UniqueName: \"kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns\") pod \"payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:32.785631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:32.785606    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:33.066202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:33.066180    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:23:33.067718 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:23:33.067690    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d1dc6a0_29ca_4fb8_985f_afcd714b727a.slice/crio-11275a7832f5e6b3a35d7ca89a722b4a24f380b4c2b73da148610e88143605f9 WatchSource:0}: Error finding container 11275a7832f5e6b3a35d7ca89a722b4a24f380b4c2b73da148610e88143605f9: Status 404 returned error can't find the container with id 11275a7832f5e6b3a35d7ca89a722b4a24f380b4c2b73da148610e88143605f9
Sep 18 20:23:33.975027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:33.974994    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" event={"ID":"2d1dc6a0-29ca-4fb8-985f-afcd714b727a","Type":"ContainerStarted","Data":"daccc9ad7d5b70eed61ca6ee5bfe5d39095c7599968b260f0e6ab4058d434a36"}
Sep 18 20:23:33.975409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:33.975049    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" event={"ID":"2d1dc6a0-29ca-4fb8-985f-afcd714b727a","Type":"ContainerStarted","Data":"11275a7832f5e6b3a35d7ca89a722b4a24f380b4c2b73da148610e88143605f9"}
Sep 18 20:23:33.975409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:33.975102    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:33.996625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:33.996584    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" podStartSLOduration=1.996570459 podStartE2EDuration="1.996570459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:23:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:23:33.993799548 +0000 UTC m=+1358.123101500" watchObservedRunningTime="2026-09-18 20:23:33.996570459 +0000 UTC m=+1358.125872409"
Sep 18 20:23:44.984878 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:44.984847    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:23:58.249954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:58.249919    2566 scope.go:117] "RemoveContainer" containerID="31b86a638bb3b2e533568923f9008679528fc58d78ebf756527e8b69afd6dd26"
Sep 18 20:23:59.710668 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.710632    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"]
Sep 18 20:23:59.714565 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.714543    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:23:59.716824 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.716793    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:23:59.716951 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.716912    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:23:59.722375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.722356    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"]
Sep 18 20:23:59.815591 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.815556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf\" (UID: \"fa3540a4-3e90-4dd7-9a81-9520e685421f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:23:59.916978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.916930    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf\" (UID: \"fa3540a4-3e90-4dd7-9a81-9520e685421f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:23:59.917575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:23:59.917553    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf\" (UID: \"fa3540a4-3e90-4dd7-9a81-9520e685421f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:24:00.030398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:00.030313    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:24:00.309143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:00.309111    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"]
Sep 18 20:24:00.310018 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:24:00.309974    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa3540a4_3e90_4dd7_9a81_9520e685421f.slice/crio-f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1 WatchSource:0}: Error finding container f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1: Status 404 returned error can't find the container with id f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1
Sep 18 20:24:01.150338 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:01.150305    2566 generic.go:356] "Generic (PLEG): container finished" podID="fa3540a4-3e90-4dd7-9a81-9520e685421f" containerID="5a593212543ec90fdefe87c7ddab8eb257ad4ded6d486932487f5c829468a7f5" exitCode=0
Sep 18 20:24:01.150700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:01.150359    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf" event={"ID":"fa3540a4-3e90-4dd7-9a81-9520e685421f","Type":"ContainerDied","Data":"5a593212543ec90fdefe87c7ddab8eb257ad4ded6d486932487f5c829468a7f5"}
Sep 18 20:24:01.150700 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:01.150411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf" event={"ID":"fa3540a4-3e90-4dd7-9a81-9520e685421f","Type":"ContainerStarted","Data":"f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1"}
Sep 18 20:24:02.157071 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.157024    2566 generic.go:356] "Generic (PLEG): container finished" podID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerID="280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894" exitCode=0
Sep 18 20:24:02.157485 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.157099    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerDied","Data":"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"}
Sep 18 20:24:02.157545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.157516    2566 scope.go:117] "RemoveContainer" containerID="280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"
Sep 18 20:24:02.158823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.158792    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:24:02.158931 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.158866    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:24:02.158931 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.158905    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:24:02.323664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.323641    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:24:02.449922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.449896    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca\") pod \"fa3540a4-3e90-4dd7-9a81-9520e685421f\" (UID: \"fa3540a4-3e90-4dd7-9a81-9520e685421f\") "
Sep 18 20:24:02.450343 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.450314    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "fa3540a4-3e90-4dd7-9a81-9520e685421f" (UID: "fa3540a4-3e90-4dd7-9a81-9520e685421f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:02.551455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:02.551370    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/fa3540a4-3e90-4dd7-9a81-9520e685421f-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:03.169387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.169345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerStarted","Data":"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"}
Sep 18 20:24:03.171113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.171088    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"
Sep 18 20:24:03.171238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.171086    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf" event={"ID":"fa3540a4-3e90-4dd7-9a81-9520e685421f","Type":"ContainerDied","Data":"f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1"}
Sep 18 20:24:03.171238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.171201    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f239fb92e10f003cca1e170d39f6bc886b87604dd6ce5d3853e4aeb97485d7f1"
Sep 18 20:24:03.629534 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.629493    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:24:03.629832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.629690    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:24:03.629832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:03.629749    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:24:04.629930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:04.629893    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:24:04.630317 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:04.629966    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:24:05.629161 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:05.629127    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:24:05.629337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:05.629197    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:24:06.629705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:06.629672    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:24:06.630100 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:06.629725    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:24:07.629641 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:07.629608    2566 patch_prober.go:28] interesting pod/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused" start-of-body=
Sep 18 20:24:07.629816 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:07.629660    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.79:15021/healthz/ready\": dial tcp 10.132.0.79:15021: connect: connection refused"
Sep 18 20:24:08.633559 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:08.633531    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:24:08.633928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:08.633913    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:24:08.634512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:08.634495    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:24:15.412427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:15.412387    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"]
Sep 18 20:24:15.414618 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:15.414595    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w3-b88002-27wbf"]
Sep 18 20:24:16.081617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:16.081575    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa3540a4-3e90-4dd7-9a81-9520e685421f" path="/var/lib/kubelet/pods/fa3540a4-3e90-4dd7-9a81-9520e685421f/volumes"
Sep 18 20:24:22.208500 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:22.208471    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:24:22.208885 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:22.208686    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy" containerID="cri-o://ed4bebbe202ea21a0484185e3ad2cc8658766c782e40813cb0d6adc56f1fccfe" gracePeriod=30
Sep 18 20:24:27.327943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.327918    2566 generic.go:356] "Generic (PLEG): container finished" podID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerID="ed4bebbe202ea21a0484185e3ad2cc8658766c782e40813cb0d6adc56f1fccfe" exitCode=0
Sep 18 20:24:27.328364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.327990    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerDied","Data":"ed4bebbe202ea21a0484185e3ad2cc8658766c782e40813cb0d6adc56f1fccfe"}
Sep 18 20:24:27.328364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.328088    2566 scope.go:117] "RemoveContainer" containerID="320e5e681308ab19f7160c44ab3921e98d68ccbf3dd3c45a634aead7b1ba97a8"
Sep 18 20:24:27.472832 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.472809    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:24:27.513348 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513279    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513476 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513359    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5dt76\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513476 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513407    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513476 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513439    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513655 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513484    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513655 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513516    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513655 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513586    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513655 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513622    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513665    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs\") pod \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\" (UID: \"d640ffa4-03df-4e5a-86c7-582d2682a1f9\") "
Sep 18 20:24:27.513874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513856    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:24:27.513990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.513860    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:27.514115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.514091    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:27.514300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.514151    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.514401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.514316    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.514401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.514117    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data" (OuterVolumeSpecName: "istio-data") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:27.514401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.514332    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:27.515635 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.515613    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76" (OuterVolumeSpecName: "kube-api-access-5dt76") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "kube-api-access-5dt76". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:27.515718 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.515646    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token" (OuterVolumeSpecName: "istio-token") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:24:27.515843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.515820    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:24:27.516150 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.516116    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d640ffa4-03df-4e5a-86c7-582d2682a1f9" (UID: "d640ffa4-03df-4e5a-86c7-582d2682a1f9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:24:27.614876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614850    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.614876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614873    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.614876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614881    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.615047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614889    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.615047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614897    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.615047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614905    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5dt76\" (UniqueName: \"kubernetes.io/projected/d640ffa4-03df-4e5a-86c7-582d2682a1f9-kube-api-access-5dt76\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:27.615047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:27.614913    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d640ffa4-03df-4e5a-86c7-582d2682a1f9-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:24:28.335540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:28.335518    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"
Sep 18 20:24:28.335911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:28.335568    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws" event={"ID":"d640ffa4-03df-4e5a-86c7-582d2682a1f9","Type":"ContainerDied","Data":"f8d8bd9d3bbd4cd531370b25feca7b2be46d6dc43c30bca4eeb94dd4804c104f"}
Sep 18 20:24:28.335911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:28.335615    2566 scope.go:117] "RemoveContainer" containerID="ed4bebbe202ea21a0484185e3ad2cc8658766c782e40813cb0d6adc56f1fccfe"
Sep 18 20:24:28.354020 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:28.353997    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:24:28.359524 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:28.359503    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w3-b88002-openshift-default-566596cf8b-8bcws"]
Sep 18 20:24:30.081436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:30.081405    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" path="/var/lib/kubelet/pods/d640ffa4-03df-4e5a-86c7-582d2682a1f9/volumes"
Sep 18 20:24:58.283266 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:58.283236    2566 scope.go:117] "RemoveContainer" containerID="1868cc67deaabb9c47413ad2c40eb050b7cad0ddf34a7715b37246e11fe2621c"
Sep 18 20:24:58.291768 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:24:58.291749    2566 scope.go:117] "RemoveContainer" containerID="587e9a0e627adb3df032bb996aa56d66adbc6415e3f77ba6a2905089ba3b397f"
Sep 18 20:25:58.156843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:25:58.156815    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:25:58.160373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:25:58.160351    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:25:58.321737 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:25:58.321705    2566 scope.go:117] "RemoveContainer" containerID="297af1590a3f24d1e740301ca83d13cc17f48b8121c858941b90e1e5f267f66f"
Sep 18 20:26:09.038752 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.038724    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"]
Sep 18 20:26:09.039216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039195    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:26:09.039216 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039211    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:26:09.039407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039225    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa3540a4-3e90-4dd7-9a81-9520e685421f" containerName="revoke-keys"
Sep 18 20:26:09.039407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039231    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa3540a4-3e90-4dd7-9a81-9520e685421f" containerName="revoke-keys"
Sep 18 20:26:09.039407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039298    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fa3540a4-3e90-4dd7-9a81-9520e685421f" containerName="revoke-keys"
Sep 18 20:26:09.039407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039307    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:26:09.039407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.039316    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:26:09.042695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.042674    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:09.045118 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.045100    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:26:09.045229 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.045175    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:26:09.050196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.050168    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"]
Sep 18 20:26:09.165565 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.165469    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc\" (UID: \"96458e5f-3b06-4c5b-858d-aada078d9f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:09.267122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.267079    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc\" (UID: \"96458e5f-3b06-4c5b-858d-aada078d9f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:09.267727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.267703    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc\" (UID: \"96458e5f-3b06-4c5b-858d-aada078d9f9f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:09.356078 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.355992    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:09.630633 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.630612    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"]
Sep 18 20:26:09.633133 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:26:09.633104    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96458e5f_3b06_4c5b_858d_aada078d9f9f.slice/crio-29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef WatchSource:0}: Error finding container 29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef: Status 404 returned error can't find the container with id 29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef
Sep 18 20:26:09.967779 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.967748    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc" event={"ID":"96458e5f-3b06-4c5b-858d-aada078d9f9f","Type":"ContainerStarted","Data":"10cd24ef46b098836907310200d6fc2e09e8b4aa0ae8789579426c23701e604c"}
Sep 18 20:26:09.967943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.967811    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc" event={"ID":"96458e5f-3b06-4c5b-858d-aada078d9f9f","Type":"ContainerStarted","Data":"29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef"}
Sep 18 20:26:09.982711 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:09.982666    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc" podStartSLOduration=0.982649606 podStartE2EDuration="982.649606ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:26:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:26:09.98183188 +0000 UTC m=+1514.111133832" watchObservedRunningTime="2026-09-18 20:26:09.982649606 +0000 UTC m=+1514.111951557"
Sep 18 20:26:10.975021 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:10.974985    2566 generic.go:356] "Generic (PLEG): container finished" podID="96458e5f-3b06-4c5b-858d-aada078d9f9f" containerID="10cd24ef46b098836907310200d6fc2e09e8b4aa0ae8789579426c23701e604c" exitCode=0
Sep 18 20:26:10.975430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:10.975087    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc" event={"ID":"96458e5f-3b06-4c5b-858d-aada078d9f9f","Type":"ContainerDied","Data":"10cd24ef46b098836907310200d6fc2e09e8b4aa0ae8789579426c23701e604c"}
Sep 18 20:26:12.117883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.117856    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:12.199540 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.199512    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca\") pod \"96458e5f-3b06-4c5b-858d-aada078d9f9f\" (UID: \"96458e5f-3b06-4c5b-858d-aada078d9f9f\") "
Sep 18 20:26:12.199886 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.199865    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "96458e5f-3b06-4c5b-858d-aada078d9f9f" (UID: "96458e5f-3b06-4c5b-858d-aada078d9f9f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:26:12.200094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.200045    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/96458e5f-3b06-4c5b-858d-aada078d9f9f-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:26:12.988827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.988792    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc" event={"ID":"96458e5f-3b06-4c5b-858d-aada078d9f9f","Type":"ContainerDied","Data":"29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef"}
Sep 18 20:26:12.988827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.988825    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"
Sep 18 20:26:12.989029 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:12.988844    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="29d4ca8caa00227a28eac5276af94df17da391a1f8d52df2150d6e2ae73afbef"
Sep 18 20:26:24.898246 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:24.898215    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:26:24.898705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:24.898427    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerName="payload-pre-processing" containerID="cri-o://daccc9ad7d5b70eed61ca6ee5bfe5d39095c7599968b260f0e6ab4058d434a36" gracePeriod=30
Sep 18 20:26:24.983546 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:24.983505    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.80:9004: connect: connection refused"
Sep 18 20:26:25.071142 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.071106    2566 generic.go:356] "Generic (PLEG): container finished" podID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerID="daccc9ad7d5b70eed61ca6ee5bfe5d39095c7599968b260f0e6ab4058d434a36" exitCode=0
Sep 18 20:26:25.071290 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.071263    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" event={"ID":"2d1dc6a0-29ca-4fb8-985f-afcd714b727a","Type":"ContainerDied","Data":"daccc9ad7d5b70eed61ca6ee5bfe5d39095c7599968b260f0e6ab4058d434a36"}
Sep 18 20:26:25.088625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.088580    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"]
Sep 18 20:26:25.093983 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.093948    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-mt-10f4b77d-jt9wc"]
Sep 18 20:26:25.147764 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.147739    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:26:25.244128 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.244037    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5mrns\" (UniqueName: \"kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns\") pod \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") "
Sep 18 20:26:25.244338 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.244267    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume\") pod \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\" (UID: \"2d1dc6a0-29ca-4fb8-985f-afcd714b727a\") "
Sep 18 20:26:25.255659 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.255598    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns" (OuterVolumeSpecName: "kube-api-access-5mrns") pod "2d1dc6a0-29ca-4fb8-985f-afcd714b727a" (UID: "2d1dc6a0-29ca-4fb8-985f-afcd714b727a"). InnerVolumeSpecName "kube-api-access-5mrns". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:26:25.255989 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.255953    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2d1dc6a0-29ca-4fb8-985f-afcd714b727a" (UID: "2d1dc6a0-29ca-4fb8-985f-afcd714b727a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:26:25.345879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.345844    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:26:25.345879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:25.345878    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5mrns\" (UniqueName: \"kubernetes.io/projected/2d1dc6a0-29ca-4fb8-985f-afcd714b727a-kube-api-access-5mrns\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:26:26.078013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.077989    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"
Sep 18 20:26:26.080960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.080935    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96458e5f-3b06-4c5b-858d-aada078d9f9f" path="/var/lib/kubelet/pods/96458e5f-3b06-4c5b-858d-aada078d9f9f/volumes"
Sep 18 20:26:26.081362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.081344    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz" event={"ID":"2d1dc6a0-29ca-4fb8-985f-afcd714b727a","Type":"ContainerDied","Data":"11275a7832f5e6b3a35d7ca89a722b4a24f380b4c2b73da148610e88143605f9"}
Sep 18 20:26:26.081428 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.081390    2566 scope.go:117] "RemoveContainer" containerID="daccc9ad7d5b70eed61ca6ee5bfe5d39095c7599968b260f0e6ab4058d434a36"
Sep 18 20:26:26.105321 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.105299    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:26:26.108685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:26.108649    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-10f4b77d-6754fff886-7rvcz"]
Sep 18 20:26:28.080913 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:26:28.080876    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" path="/var/lib/kubelet/pods/2d1dc6a0-29ca-4fb8-985f-afcd714b727a/volumes"
Sep 18 20:27:13.561279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561248    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"]
Sep 18 20:27:13.561798 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561781    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerName="payload-pre-processing"
Sep 18 20:27:13.561843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561799    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerName="payload-pre-processing"
Sep 18 20:27:13.561843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561813    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:27:13.561843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561821    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="d640ffa4-03df-4e5a-86c7-582d2682a1f9" containerName="istio-proxy"
Sep 18 20:27:13.561843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561832    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96458e5f-3b06-4c5b-858d-aada078d9f9f" containerName="revoke-keys"
Sep 18 20:27:13.561843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561838    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="96458e5f-3b06-4c5b-858d-aada078d9f9f" containerName="revoke-keys"
Sep 18 20:27:13.562009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561902    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2d1dc6a0-29ca-4fb8-985f-afcd714b727a" containerName="payload-pre-processing"
Sep 18 20:27:13.562009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.561912    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="96458e5f-3b06-4c5b-858d-aada078d9f9f" containerName="revoke-keys"
Sep 18 20:27:13.564629 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.564615    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:13.566834 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.566813    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:27:13.566929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.566918    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:27:13.571999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.571969    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"]
Sep 18 20:27:13.707373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.707344    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn\" (UID: \"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:13.808561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.808536    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn\" (UID: \"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:13.809115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.809098    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn\" (UID: \"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:13.876420 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:13.876370    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:14.153296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:14.153266    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"]
Sep 18 20:27:14.154491 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:27:14.154262    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod419c5fca_408c_4acb_a9e9_2f7c6fc3b77f.slice/crio-ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1 WatchSource:0}: Error finding container ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1: Status 404 returned error can't find the container with id ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1
Sep 18 20:27:14.355924 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:14.355892    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn" event={"ID":"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f","Type":"ContainerStarted","Data":"ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1"}
Sep 18 20:27:15.362789 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:15.362755    2566 generic.go:356] "Generic (PLEG): container finished" podID="419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" containerID="8968d81d9fa9bade9f634bb6323ca3e2c374381237603b88554973750af60670" exitCode=0
Sep 18 20:27:15.363226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:15.362801    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn" event={"ID":"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f","Type":"ContainerDied","Data":"8968d81d9fa9bade9f634bb6323ca3e2c374381237603b88554973750af60670"}
Sep 18 20:27:16.563682 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.563651    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:16.586176 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.586151    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:27:16.586362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.586342    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy" containerID="cri-o://3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681" gracePeriod=30
Sep 18 20:27:16.662904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.662833    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca\") pod \"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f\" (UID: \"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f\") "
Sep 18 20:27:16.663271 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.663247    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" (UID: "419c5fca-408c-4acb-a9e9-2f7c6fc3b77f"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:16.663357 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:16.663345    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:17.376094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:17.376046    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn" event={"ID":"419c5fca-408c-4acb-a9e9-2f7c6fc3b77f","Type":"ContainerDied","Data":"ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1"}
Sep 18 20:27:17.376094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:17.376079    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"
Sep 18 20:27:17.376289 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:17.376108    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="ac4e3809fa8f4668be5d7fc7ede0860e981cc861b269631bb42e6468d292b6a1"
Sep 18 20:27:18.436780 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:18.436747    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:27:21.850378 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:21.850322    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:27:22.016316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016292    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016354    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016390    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hvksd\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016420    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016451    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016551    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:22.016620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016589    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016626    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016652    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016695    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs\") pod \"62f64ad1-6593-43e9-934b-eea2ed635cbd\" (UID: \"62f64ad1-6593-43e9-934b-eea2ed635cbd\") "
Sep 18 20:27:22.016862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016719    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:22.016918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016862    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data" (OuterVolumeSpecName: "istio-data") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:22.016973 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.016921    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:22.017270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.017244    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.017270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.017269    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.017429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.017282    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.017429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.017295    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.017429 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.017324    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:22.018754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.018729    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd" (OuterVolumeSpecName: "kube-api-access-hvksd") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "kube-api-access-hvksd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:22.018754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.018742    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token" (OuterVolumeSpecName: "istio-token") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:22.018998 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.018979    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:27:22.019047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.018977    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "62f64ad1-6593-43e9-934b-eea2ed635cbd" (UID: "62f64ad1-6593-43e9-934b-eea2ed635cbd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:22.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.118484    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.118507    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hvksd\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-kube-api-access-hvksd\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.118516    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.118524    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/62f64ad1-6593-43e9-934b-eea2ed635cbd-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.118539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.118532    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/62f64ad1-6593-43e9-934b-eea2ed635cbd-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:22.408014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.407937    2566 generic.go:356] "Generic (PLEG): container finished" podID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerID="3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681" exitCode=0
Sep 18 20:27:22.408014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.407987    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"
Sep 18 20:27:22.408184 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.408020    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerDied","Data":"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"}
Sep 18 20:27:22.408184 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.408106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq" event={"ID":"62f64ad1-6593-43e9-934b-eea2ed635cbd","Type":"ContainerDied","Data":"d0f9ce20ca483799d8c301929070540509f6be4eb63db59f1e5fe4eff5038e29"}
Sep 18 20:27:22.408184 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.408132    2566 scope.go:117] "RemoveContainer" containerID="3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"
Sep 18 20:27:22.418283 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.418254    2566 scope.go:117] "RemoveContainer" containerID="280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"
Sep 18 20:27:22.425791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.425768    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:27:22.429203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.429172    2566 scope.go:117] "RemoveContainer" containerID="3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"
Sep 18 20:27:22.429493 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:22.429466    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681\": container with ID starting with 3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681 not found: ID does not exist" containerID="3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"
Sep 18 20:27:22.429587 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.429502    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681"} err="failed to get container status \"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681\": rpc error: code = NotFound desc = could not find container \"3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681\": container with ID starting with 3e7d0dc12cf8161c3e52cb9b80e4f535c1765463b5afdd7d6e56dbcd04c40681 not found: ID does not exist"
Sep 18 20:27:22.429587 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.429526    2566 scope.go:117] "RemoveContainer" containerID="280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"
Sep 18 20:27:22.429788 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:22.429768    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894\": container with ID starting with 280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894 not found: ID does not exist" containerID="280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"
Sep 18 20:27:22.429839 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.429793    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894"} err="failed to get container status \"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894\": rpc error: code = NotFound desc = could not find container \"280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894\": container with ID starting with 280c972d30e734927d9b471d6c7d899bd07da8cbaee75bc6b11666b4b3268894 not found: ID does not exist"
Sep 18 20:27:22.432041 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:22.432022    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-10f4b77d-openshift-default-745cc64984-7dxmq"]
Sep 18 20:27:24.081346 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:24.081309    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" path="/var/lib/kubelet/pods/62f64ad1-6593-43e9-934b-eea2ed635cbd/volumes"
Sep 18 20:27:29.069571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.069529    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:27:29.070070 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.069881    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" podUID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" containerName="payload-pre-processing" containerID="cri-o://8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6" gracePeriod=30
Sep 18 20:27:29.253557 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.253529    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"]
Sep 18 20:27:29.257335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.257306    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-worker-w0-116778-q7gsn"]
Sep 18 20:27:29.359986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.359960    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:27:29.391938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.391912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5grhg\" (UniqueName: \"kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg\") pod \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") "
Sep 18 20:27:29.392106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.392088    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume\") pod \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\" (UID: \"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675\") "
Sep 18 20:27:29.392456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.392432    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" (UID: "4b2f25b8-4b44-4a9e-bbf7-a4f20996d675"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:29.394138 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.394107    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg" (OuterVolumeSpecName: "kube-api-access-5grhg") pod "4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" (UID: "4b2f25b8-4b44-4a9e-bbf7-a4f20996d675"). InnerVolumeSpecName "kube-api-access-5grhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:29.450555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.450524    2566 generic.go:356] "Generic (PLEG): container finished" podID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" containerID="8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6" exitCode=0
Sep 18 20:27:29.450726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.450577    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" event={"ID":"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675","Type":"ContainerDied","Data":"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"}
Sep 18 20:27:29.450726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.450579    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"
Sep 18 20:27:29.450726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.450618    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv" event={"ID":"4b2f25b8-4b44-4a9e-bbf7-a4f20996d675","Type":"ContainerDied","Data":"d912297c8f2063c563b798ae74f2cd9775851c11c4808a87097a38556be58c60"}
Sep 18 20:27:29.450726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.450640    2566 scope.go:117] "RemoveContainer" containerID="8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"
Sep 18 20:27:29.459885 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.459867    2566 scope.go:117] "RemoveContainer" containerID="8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"
Sep 18 20:27:29.460165 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:29.460146    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6\": container with ID starting with 8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6 not found: ID does not exist" containerID="8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"
Sep 18 20:27:29.460237 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.460178    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6"} err="failed to get container status \"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6\": rpc error: code = NotFound desc = could not find container \"8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6\": container with ID starting with 8525ffd4a79eb47ce6b9f5f1fcca46366bc9f273aea30e8bed31c41d3155ede6 not found: ID does not exist"
Sep 18 20:27:29.480628 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.480543    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:27:29.493325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.493293    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:29.493325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.493320    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5grhg\" (UniqueName: \"kubernetes.io/projected/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675-kube-api-access-5grhg\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:29.516141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:29.516111    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-worker-w0-116778-79c5f6864b-2zsqv"]
Sep 18 20:27:30.081618 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:30.081590    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" path="/var/lib/kubelet/pods/419c5fca-408c-4acb-a9e9-2f7c6fc3b77f/volumes"
Sep 18 20:27:30.081967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:30.081948    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" path="/var/lib/kubelet/pods/4b2f25b8-4b44-4a9e-bbf7-a4f20996d675/volumes"
Sep 18 20:27:33.456560 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.456521    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" podUID="8743f4d8-9cef-450e-a51c-53813857640a" containerName="main" containerID="cri-o://2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f" gracePeriod=45
Sep 18 20:27:33.712559 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.712504    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:27:33.833359 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833331    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6lv9m\" (UniqueName: \"kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833415    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833444    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833493    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833693 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833545    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache\") pod \"8743f4d8-9cef-450e-a51c-53813857640a\" (UID: \"8743f4d8-9cef-450e-a51c-53813857640a\") "
Sep 18 20:27:33.833807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833713    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home" (OuterVolumeSpecName: "home") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.833807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833790    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.833948 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.833920    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache" (OuterVolumeSpecName: "model-cache") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.834027 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.834006    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-tmp-dir\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.834111 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.834026    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-home\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.835754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.835722    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm" (OuterVolumeSpecName: "dshm") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:33.835754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.835745    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m" (OuterVolumeSpecName: "kube-api-access-6lv9m") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "kube-api-access-6lv9m". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:33.835900 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.835799    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "8743f4d8-9cef-450e-a51c-53813857640a" (UID: "8743f4d8-9cef-450e-a51c-53813857640a"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:27:33.934817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.934788    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6lv9m\" (UniqueName: \"kubernetes.io/projected/8743f4d8-9cef-450e-a51c-53813857640a-kube-api-access-6lv9m\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.934817 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.934814    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-dshm\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.934999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.934825    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8743f4d8-9cef-450e-a51c-53813857640a-tls-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:33.934999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:33.934835    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/8743f4d8-9cef-450e-a51c-53813857640a-model-cache\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:34.484342 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.484308    2566 generic.go:356] "Generic (PLEG): container finished" podID="8743f4d8-9cef-450e-a51c-53813857640a" containerID="2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f" exitCode=0
Sep 18 20:27:34.484719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.484357    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" event={"ID":"8743f4d8-9cef-450e-a51c-53813857640a","Type":"ContainerDied","Data":"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"}
Sep 18 20:27:34.484719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.484389    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp" event={"ID":"8743f4d8-9cef-450e-a51c-53813857640a","Type":"ContainerDied","Data":"afa72153b2b4c2ed6611da2dc12d8f0b94dace2c1cc0c38daf5e2f8042b55c3a"}
Sep 18 20:27:34.484719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.484408    2566 scope.go:117] "RemoveContainer" containerID="2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"
Sep 18 20:27:34.484719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.484359    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"
Sep 18 20:27:34.493755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.493735    2566 scope.go:117] "RemoveContainer" containerID="2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"
Sep 18 20:27:34.494045 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:34.494024    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f\": container with ID starting with 2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f not found: ID does not exist" containerID="2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"
Sep 18 20:27:34.494227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.494051    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f"} err="failed to get container status \"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f\": rpc error: code = NotFound desc = could not find container \"2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f\": container with ID starting with 2a7666a3367c26ec4d08b8403703635a3248989e9c5cb14d6a068ab1beb45a5f not found: ID does not exist"
Sep 18 20:27:34.502542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.502518    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:27:34.509008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:34.508979    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-116778/distinct-2-w0-116778-kserve-74969f55f8-shrxp"]
Sep 18 20:27:35.506375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:35.506344    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:27:35.506849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:35.506597    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy" containerID="cri-o://04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb" gracePeriod=30
Sep 18 20:27:36.082535 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:36.082501    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8743f4d8-9cef-450e-a51c-53813857640a" path="/var/lib/kubelet/pods/8743f4d8-9cef-450e-a51c-53813857640a/volumes"
Sep 18 20:27:40.772219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.772195    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:27:40.906845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.906771    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.906845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.906810    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.906858    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.906881    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907028 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.906912    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907027    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907082    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907154    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-89ghg\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907180    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907209    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket\") pod \"432fe170-b3b7-45a8-a573-5d45a98a8f96\" (UID: \"432fe170-b3b7-45a8-a573-5d45a98a8f96\") "
Sep 18 20:27:40.907215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907207    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data" (OuterVolumeSpecName: "istio-data") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:40.907558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907255    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:27:40.907558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907409    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:40.907558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907529    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:40.907727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907663    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:40.907727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907684    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:40.907727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907698    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:40.907727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907710    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/432fe170-b3b7-45a8-a573-5d45a98a8f96-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:40.907727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.907722    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:40.909264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.909240    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token" (OuterVolumeSpecName: "istio-token") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:40.909264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.909242    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:27:40.909419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.909268    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:27:40.909419 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:40.909301    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg" (OuterVolumeSpecName: "kube-api-access-89ghg") pod "432fe170-b3b7-45a8-a573-5d45a98a8f96" (UID: "432fe170-b3b7-45a8-a573-5d45a98a8f96"). InnerVolumeSpecName "kube-api-access-89ghg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:27:41.008897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.008869    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:41.008897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.008894    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-89ghg\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-kube-api-access-89ghg\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:41.009050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.008904    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:41.009050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.008913    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/432fe170-b3b7-45a8-a573-5d45a98a8f96-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:27:41.531372 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.531338    2566 generic.go:356] "Generic (PLEG): container finished" podID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerID="04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb" exitCode=0
Sep 18 20:27:41.531588 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.531389    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"
Sep 18 20:27:41.531588 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.531424    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerDied","Data":"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"}
Sep 18 20:27:41.531588 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.531489    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl" event={"ID":"432fe170-b3b7-45a8-a573-5d45a98a8f96","Type":"ContainerDied","Data":"5a2feae6f852483b5548a3044eeb5caaa7aa401c5ce5dfdbb05d06a52f057bdf"}
Sep 18 20:27:41.531588 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.531520    2566 scope.go:117] "RemoveContainer" containerID="04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"
Sep 18 20:27:41.542960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.542941    2566 scope.go:117] "RemoveContainer" containerID="fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"
Sep 18 20:27:41.553308 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.553285    2566 scope.go:117] "RemoveContainer" containerID="04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"
Sep 18 20:27:41.553648 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:41.553627    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb\": container with ID starting with 04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb not found: ID does not exist" containerID="04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"
Sep 18 20:27:41.553742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.553656    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb"} err="failed to get container status \"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb\": rpc error: code = NotFound desc = could not find container \"04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb\": container with ID starting with 04838a8172be0e177251e56af2611d3ebabdf9434a30b205932be04172b415cb not found: ID does not exist"
Sep 18 20:27:41.553742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.553677    2566 scope.go:117] "RemoveContainer" containerID="fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"
Sep 18 20:27:41.553926 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:27:41.553911    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c\": container with ID starting with fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c not found: ID does not exist" containerID="fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"
Sep 18 20:27:41.553994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.553929    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c"} err="failed to get container status \"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c\": rpc error: code = NotFound desc = could not find container \"fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c\": container with ID starting with fecd68e7cd587864a450bf6397478099d6b6d87e357927712d5f8a93094f9b2c not found: ID does not exist"
Sep 18 20:27:41.554498 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.554476    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:27:41.557275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:41.557255    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-116778-openshift-default-6b75888f84-g7fvl"]
Sep 18 20:27:42.080937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:42.080902    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" path="/var/lib/kubelet/pods/432fe170-b3b7-45a8-a573-5d45a98a8f96/volumes"
Sep 18 20:27:58.355732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:27:58.355700    2566 scope.go:117] "RemoveContainer" containerID="efb6f02f0153a6daed1badcdb3d7310f9bdd2111348f8cf024539ecb6584a9b4"
Sep 18 20:28:10.434341 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434307    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:28:10.434845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434827    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:10.434845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434843    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434858    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" containerName="revoke-keys"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434864    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" containerName="revoke-keys"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434880    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434888    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434900    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8743f4d8-9cef-450e-a51c-53813857640a" containerName="main"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434905    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="8743f4d8-9cef-450e-a51c-53813857640a" containerName="main"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434915    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434920    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434927    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.434929 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434932    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434942    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" containerName="payload-pre-processing"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.434949    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" containerName="payload-pre-processing"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435016    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435024    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435031    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="62f64ad1-6593-43e9-934b-eea2ed635cbd" containerName="istio-proxy"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435040    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4b2f25b8-4b44-4a9e-bbf7-a4f20996d675" containerName="payload-pre-processing"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435050    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="419c5fca-408c-4acb-a9e9-2f7c6fc3b77f" containerName="revoke-keys"
Sep 18 20:28:10.435260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.435070    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="8743f4d8-9cef-450e-a51c-53813857640a" containerName="main"
Sep 18 20:28:10.446208 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.446183    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.448889 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.448687    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-a-77f5c1-openshift-default-dockercfg-b7zf6\""
Sep 18 20:28:10.451511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.451476    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:28:10.506981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.506943    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.506982    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pn8pb\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507021    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507094    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507137    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507246    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507274    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507383 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507313    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.507488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.507412    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.607926 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.607893    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.607926 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.607925    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608069    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608203    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608226    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pn8pb\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608267    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608330    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608357    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608378    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608404    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608524    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.608755 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.608673    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.609151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.609127    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.610640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.610621    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.610962 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.610945    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.615999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.615977    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.616211 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.616192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pn8pb\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb\") pod \"e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") " pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:10.760486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:10.760451    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:11.041461 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.041404    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:28:11.042986 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:28:11.042954    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f6b1f43_0a89_412c_8bb2_74c94b69a9a9.slice/crio-6300e37a3d73593b34628163e8521244452c7c014b9183bc79de70e54751c091 WatchSource:0}: Error finding container 6300e37a3d73593b34628163e8521244452c7c014b9183bc79de70e54751c091: Status 404 returned error can't find the container with id 6300e37a3d73593b34628163e8521244452c7c014b9183bc79de70e54751c091
Sep 18 20:28:11.045990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.045947    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:11.046108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.046041    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:11.046182 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.046113    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:11.714200 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.714159    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerStarted","Data":"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"}
Sep 18 20:28:11.714648 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.714212    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerStarted","Data":"6300e37a3d73593b34628163e8521244452c7c014b9183bc79de70e54751c091"}
Sep 18 20:28:11.741711 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.741642    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podStartSLOduration=1.741621421 podStartE2EDuration="1.741621421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:11.73909386 +0000 UTC m=+1635.868395803" watchObservedRunningTime="2026-09-18 20:28:11.741621421 +0000 UTC m=+1635.870923372"
Sep 18 20:28:11.760733 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:11.760704    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:12.766511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:12.766483    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:13.728200 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:13.728099    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:13.730736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:13.730712    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:29.769207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:29.769166    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 20:28:29.769742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:29.769245    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:28:30.827845 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.827813    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerID="dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22" exitCode=0
Sep 18 20:28:30.828397 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.827892    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerDied","Data":"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"}
Sep 18 20:28:30.828516 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.828498    2566 scope.go:117] "RemoveContainer" containerID="dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"
Sep 18 20:28:30.829663 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.829639    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:30.829730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.829703    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:30.829771 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:30.829752    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:31.835359 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:31.835323    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerStarted","Data":"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"}
Sep 18 20:28:32.760912 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:32.760881    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:32.761291 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:32.761266    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 20:28:32.761473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:32.761447    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 20:28:33.761387 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:33.761346    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 20:28:33.761850 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:33.761410    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 20:28:34.761124 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:34.761082    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 20:28:34.761323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:34.761152    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 20:28:35.761241 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:35.761197    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 20:28:35.761696 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:35.761265    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 20:28:36.761482 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:36.761447    2566 patch_prober.go:28] interesting pod/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused" start-of-body=
Sep 18 20:28:36.762092 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:36.762040    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.84:15021/healthz/ready\": dial tcp 10.132.0.84:15021: connect: connection refused"
Sep 18 20:28:37.765161 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:37.765129    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:37.765571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:37.765480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:37.766251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:37.766232    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:28:44.328561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.328528    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:28:44.329122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.329101    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="432fe170-b3b7-45a8-a573-5d45a98a8f96" containerName="istio-proxy"
Sep 18 20:28:44.332654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.332633    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.334942 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.334926    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-api-b-102c83-openshift-default-dockercfg-z65wj\""
Sep 18 20:28:44.345094 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.345051    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:28:44.469327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469298    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469347    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469395    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469437    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469463    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gxxd4\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469687 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469560    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469687 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469660    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469782 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469688    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.469782 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.469733    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570629 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570593    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570636    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570661    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570678    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570791    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gxxd4\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570829    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.570932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570919    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.570950    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571009    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571173    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571219 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571190    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571369 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571341    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571369 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571358    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.571835 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.571815    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.572997 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.572978    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.573465 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.573442    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.579454 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.579401    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.579852 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.579835    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gxxd4\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4\") pod \"e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") " pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.645039 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.645022    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:44.921865 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.921836    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:28:44.926299 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:28:44.926269    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1609fa2_bad7_48d7_a870_e7250f052786.slice/crio-d1b1e54d3577af8155de3f10cbff72132fd07bb0d7b7000d0b997bb2edc520ab WatchSource:0}: Error finding container d1b1e54d3577af8155de3f10cbff72132fd07bb0d7b7000d0b997bb2edc520ab: Status 404 returned error can't find the container with id d1b1e54d3577af8155de3f10cbff72132fd07bb0d7b7000d0b997bb2edc520ab
Sep 18 20:28:44.929424 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.929392    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:44.929529 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.929473    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:44.929529 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:44.929519    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:28:45.936649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:45.936617    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerStarted","Data":"c79b197aaf4d7bd54ae1b0bfb1103d1377047a1962a81754eafa63225f555f8e"}
Sep 18 20:28:45.937000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:45.936668    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerStarted","Data":"d1b1e54d3577af8155de3f10cbff72132fd07bb0d7b7000d0b997bb2edc520ab"}
Sep 18 20:28:45.957889 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:45.957846    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podStartSLOduration=1.957831946 podStartE2EDuration="1.957831946s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:45.954840275 +0000 UTC m=+1670.084142225" watchObservedRunningTime="2026-09-18 20:28:45.957831946 +0000 UTC m=+1670.087133896"
Sep 18 20:28:46.645521 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:46.645489    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:46.650723 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:46.650696    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:46.943710 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:46.943604    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:46.944672 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:46.944643    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:28:51.966775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:51.966739    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:28:51.970810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:51.970787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:51.973252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:51.973230    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-102c83\""
Sep 18 20:28:51.978365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:51.978341    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:28:52.045943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.045907    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-545tg\" (UniqueName: \"kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.046201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.046180    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.147719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.147674    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-545tg\" (UniqueName: \"kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.147906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.147881    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.148595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.148573    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.156033 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.156013    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-545tg\" (UniqueName: \"kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg\") pod \"payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.283856 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.283787    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.553799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.553770    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:28:52.555198 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:28:52.555168    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89e1fa92_fba8_4319_b525_8f040e858acc.slice/crio-4d889ffbe98c12e2e30baee952167fdb0173217eeee40d0bcd5b1ce663421d06 WatchSource:0}: Error finding container 4d889ffbe98c12e2e30baee952167fdb0173217eeee40d0bcd5b1ce663421d06: Status 404 returned error can't find the container with id 4d889ffbe98c12e2e30baee952167fdb0173217eeee40d0bcd5b1ce663421d06
Sep 18 20:28:52.980902 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.980861    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" event={"ID":"89e1fa92-fba8-4319-b525-8f040e858acc","Type":"ContainerStarted","Data":"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"}
Sep 18 20:28:52.981336 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.980928    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" event={"ID":"89e1fa92-fba8-4319-b525-8f040e858acc","Type":"ContainerStarted","Data":"4d889ffbe98c12e2e30baee952167fdb0173217eeee40d0bcd5b1ce663421d06"}
Sep 18 20:28:52.981336 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.980955    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:28:52.997463 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:52.997414    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" podStartSLOduration=1.9973998800000001 podStartE2EDuration="1.99739988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:28:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:28:52.995924343 +0000 UTC m=+1677.125226297" watchObservedRunningTime="2026-09-18 20:28:52.99739988 +0000 UTC m=+1677.126701831"
Sep 18 20:28:58.394637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:58.394608    2566 scope.go:117] "RemoveContainer" containerID="7beb072ab58b96ac575ce75e77446283746bdf9d6599acb64fa2369ddd469ce4"
Sep 18 20:28:58.403254 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:28:58.403225    2566 scope.go:117] "RemoveContainer" containerID="79c26ec5f6b6cbcf96e5707c0cd6b9f3516789aefbd869e64227cad79d0be435"
Sep 18 20:29:03.988260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:03.988228    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:29:11.111325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.111289    2566 generic.go:356] "Generic (PLEG): container finished" podID="b1609fa2-bad7-48d7-a870-e7250f052786" containerID="c79b197aaf4d7bd54ae1b0bfb1103d1377047a1962a81754eafa63225f555f8e" exitCode=0
Sep 18 20:29:11.111759 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.111363    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerDied","Data":"c79b197aaf4d7bd54ae1b0bfb1103d1377047a1962a81754eafa63225f555f8e"}
Sep 18 20:29:11.111816 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.111802    2566 scope.go:117] "RemoveContainer" containerID="c79b197aaf4d7bd54ae1b0bfb1103d1377047a1962a81754eafa63225f555f8e"
Sep 18 20:29:11.113106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.113080    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:29:11.113189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.113149    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:29:11.113189 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:11.113179    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:29:12.118991 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:12.118958    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerStarted","Data":"2a55befb1d3728b1aa444d7275725b4525c72553be3f7da76192dfc347aee873"}
Sep 18 20:29:12.646537 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:12.646501    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:29:12.646954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:12.646922    2566 patch_prober.go:28] interesting pod/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 20:29:12.647200 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:12.647168    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 20:29:13.645685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:13.645648    2566 patch_prober.go:28] interesting pod/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 20:29:13.646297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:13.646263    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 20:29:14.646491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:14.646452    2566 patch_prober.go:28] interesting pod/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 20:29:14.647143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:14.647108    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 20:29:15.646544 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:15.646046    2566 patch_prober.go:28] interesting pod/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 20:29:15.646544 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:15.646130    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 20:29:16.646014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:16.645961    2566 patch_prober.go:28] interesting pod/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused" start-of-body=
Sep 18 20:29:16.646236 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:16.646077    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.85:15021/healthz/ready\": dial tcp 10.132.0.85:15021: connect: connection refused"
Sep 18 20:29:16.944256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:16.944153    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:29:17.649320 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:17.649290    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:29:17.650305 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:17.650287    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:29:20.698711 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.698678    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"]
Sep 18 20:29:20.702024 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.702008    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:20.704672 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.704652    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:29:20.704672 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.704661    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:29:20.712881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.712862    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"]
Sep 18 20:29:20.753474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.753450    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2\" (UID: \"240390ca-17d5-4ea0-9813-bb0e69ea96fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:20.854683 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.854655    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2\" (UID: \"240390ca-17d5-4ea0-9813-bb0e69ea96fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:20.855510 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:20.855486    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2\" (UID: \"240390ca-17d5-4ea0-9813-bb0e69ea96fe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:21.013137 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:21.013107    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:21.289694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:21.289666    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"]
Sep 18 20:29:21.292576 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:29:21.292550    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod240390ca_17d5_4ea0_9813_bb0e69ea96fe.slice/crio-f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d WatchSource:0}: Error finding container f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d: Status 404 returned error can't find the container with id f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d
Sep 18 20:29:22.189403 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:22.189367    2566 generic.go:356] "Generic (PLEG): container finished" podID="240390ca-17d5-4ea0-9813-bb0e69ea96fe" containerID="973ca71a49b06ed1158747a6b5089c2f7ee6fae3ebcbe9c850124291f0b7bbe6" exitCode=0
Sep 18 20:29:22.189826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:22.189440    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2" event={"ID":"240390ca-17d5-4ea0-9813-bb0e69ea96fe","Type":"ContainerDied","Data":"973ca71a49b06ed1158747a6b5089c2f7ee6fae3ebcbe9c850124291f0b7bbe6"}
Sep 18 20:29:22.189826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:22.189494    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2" event={"ID":"240390ca-17d5-4ea0-9813-bb0e69ea96fe","Type":"ContainerStarted","Data":"f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d"}
Sep 18 20:29:23.354200 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:23.354179    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:23.482506 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:23.482421    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca\") pod \"240390ca-17d5-4ea0-9813-bb0e69ea96fe\" (UID: \"240390ca-17d5-4ea0-9813-bb0e69ea96fe\") "
Sep 18 20:29:23.482814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:23.482790    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "240390ca-17d5-4ea0-9813-bb0e69ea96fe" (UID: "240390ca-17d5-4ea0-9813-bb0e69ea96fe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:29:23.583549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:23.583516    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/240390ca-17d5-4ea0-9813-bb0e69ea96fe-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:24.212112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:24.212084    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"
Sep 18 20:29:24.212112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:24.212106    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2" event={"ID":"240390ca-17d5-4ea0-9813-bb0e69ea96fe","Type":"ContainerDied","Data":"f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d"}
Sep 18 20:29:24.212329 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:24.212154    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f8db1b2857b9a4d1061d463c48f5a219d26d009ec8216339f7e4e9a40594eb5d"
Sep 18 20:29:36.419703 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:36.419666    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"]
Sep 18 20:29:36.424299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:36.424273    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-77f5c1-6sll2"]
Sep 18 20:29:38.081568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:38.081526    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="240390ca-17d5-4ea0-9813-bb0e69ea96fe" path="/var/lib/kubelet/pods/240390ca-17d5-4ea0-9813-bb0e69ea96fe/volumes"
Sep 18 20:29:42.843805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:42.843754    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:29:42.847089 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:42.844391    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy" containerID="cri-o://9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed" gracePeriod=30
Sep 18 20:29:43.174392 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.174317    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"]
Sep 18 20:29:43.174784 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.174772    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="240390ca-17d5-4ea0-9813-bb0e69ea96fe" containerName="revoke-keys"
Sep 18 20:29:43.174828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.174786    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="240390ca-17d5-4ea0-9813-bb0e69ea96fe" containerName="revoke-keys"
Sep 18 20:29:43.174870 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.174860    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="240390ca-17d5-4ea0-9813-bb0e69ea96fe" containerName="revoke-keys"
Sep 18 20:29:43.179160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.179144    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:43.182109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.181837    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:29:43.182109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.181880    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:29:43.185416 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.185392    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"]
Sep 18 20:29:43.191425 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.191400    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-102c83-ncxpq\" (UID: \"b4114aaf-276a-465c-aeab-53f70af7adc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:43.292250 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.292225    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-102c83-ncxpq\" (UID: \"b4114aaf-276a-465c-aeab-53f70af7adc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:43.292837 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.292819    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-b-102c83-ncxpq\" (UID: \"b4114aaf-276a-465c-aeab-53f70af7adc7\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:43.490335 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.490315    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:43.766477 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:43.766449    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"]
Sep 18 20:29:43.768242 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:29:43.768216    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4114aaf_276a_465c_aeab_53f70af7adc7.slice/crio-0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad WatchSource:0}: Error finding container 0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad: Status 404 returned error can't find the container with id 0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad
Sep 18 20:29:44.333450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:44.333405    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq" event={"ID":"b4114aaf-276a-465c-aeab-53f70af7adc7","Type":"ContainerStarted","Data":"ff6793181a642d73501717c897a7b907822d2287e395f0ff7d0f40a1a59daa91"}
Sep 18 20:29:44.333799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:44.333464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq" event={"ID":"b4114aaf-276a-465c-aeab-53f70af7adc7","Type":"ContainerStarted","Data":"0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad"}
Sep 18 20:29:44.349212 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:44.349170    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq" podStartSLOduration=1.349149624 podStartE2EDuration="1.349149624s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:29:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:29:44.347998448 +0000 UTC m=+1728.477300420" watchObservedRunningTime="2026-09-18 20:29:44.349149624 +0000 UTC m=+1728.478451574"
Sep 18 20:29:45.341157 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:45.341113    2566 generic.go:356] "Generic (PLEG): container finished" podID="b4114aaf-276a-465c-aeab-53f70af7adc7" containerID="ff6793181a642d73501717c897a7b907822d2287e395f0ff7d0f40a1a59daa91" exitCode=0
Sep 18 20:29:45.341547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:45.341180    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq" event={"ID":"b4114aaf-276a-465c-aeab-53f70af7adc7","Type":"ContainerDied","Data":"ff6793181a642d73501717c897a7b907822d2287e395f0ff7d0f40a1a59daa91"}
Sep 18 20:29:46.484692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:46.484672    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:46.526781 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:46.526756    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca\") pod \"b4114aaf-276a-465c-aeab-53f70af7adc7\" (UID: \"b4114aaf-276a-465c-aeab-53f70af7adc7\") "
Sep 18 20:29:46.527123 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:46.527100    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "b4114aaf-276a-465c-aeab-53f70af7adc7" (UID: "b4114aaf-276a-465c-aeab-53f70af7adc7"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:29:46.527279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:46.527268    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/b4114aaf-276a-465c-aeab-53f70af7adc7-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:47.363446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:47.363414    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"
Sep 18 20:29:47.363612 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:47.363411    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq" event={"ID":"b4114aaf-276a-465c-aeab-53f70af7adc7","Type":"ContainerDied","Data":"0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad"}
Sep 18 20:29:47.363612 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:47.363580    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0adb6f95fcaefe8ed3ae33d2cc5bbfac394b6df8397a01c2c7be6639b3fa87ad"
Sep 18 20:29:48.116762 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.116707    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:29:48.144409 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144350    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.144757 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144734    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.144880 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144795    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.144880 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144851    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.145617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144882    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.145617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144913    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pn8pb\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.145617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144964    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.145617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144987    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.145617 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.145353    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket\") pod \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\" (UID: \"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9\") "
Sep 18 20:29:48.146013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.144633    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:48.146013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.145850    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.146013 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.145906    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:29:48.146737 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.146712    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:48.147503 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.147426    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:48.147828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.147807    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data" (OuterVolumeSpecName: "istio-data") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:48.148369 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.148342    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:29:48.148608 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.148585    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token" (OuterVolumeSpecName: "istio-token") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:48.148892 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.148866    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:29:48.150286 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.150255    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb" (OuterVolumeSpecName: "kube-api-access-pn8pb") pod "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" (UID: "2f6b1f43-0a89-412c-8bb2-74c94b69a9a9"). InnerVolumeSpecName "kube-api-access-pn8pb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:48.246849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246809    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pn8pb\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-kube-api-access-pn8pb\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.246849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246849    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246863    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246878    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246891    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246932    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246945    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.247097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.246956    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:48.370980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.370901    2566 generic.go:356] "Generic (PLEG): container finished" podID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerID="9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed" exitCode=0
Sep 18 20:29:48.370980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.370944    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"
Sep 18 20:29:48.371233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.370986    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerDied","Data":"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"}
Sep 18 20:29:48.371233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.371041    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5" event={"ID":"2f6b1f43-0a89-412c-8bb2-74c94b69a9a9","Type":"ContainerDied","Data":"6300e37a3d73593b34628163e8521244452c7c014b9183bc79de70e54751c091"}
Sep 18 20:29:48.371233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.371096    2566 scope.go:117] "RemoveContainer" containerID="9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"
Sep 18 20:29:48.380936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.380919    2566 scope.go:117] "RemoveContainer" containerID="dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"
Sep 18 20:29:48.389482 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.389464    2566 scope.go:117] "RemoveContainer" containerID="9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"
Sep 18 20:29:48.389732 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:29:48.389715    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed\": container with ID starting with 9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed not found: ID does not exist" containerID="9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"
Sep 18 20:29:48.389773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.389740    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed"} err="failed to get container status \"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed\": rpc error: code = NotFound desc = could not find container \"9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed\": container with ID starting with 9c4aa051b611804e4dccd640bc824b77aebf2a9f474ea03a34fcd6c7312194ed not found: ID does not exist"
Sep 18 20:29:48.389773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.389755    2566 scope.go:117] "RemoveContainer" containerID="dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"
Sep 18 20:29:48.390014 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:29:48.390000    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22\": container with ID starting with dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22 not found: ID does not exist" containerID="dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"
Sep 18 20:29:48.390051 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.390019    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22"} err="failed to get container status \"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22\": rpc error: code = NotFound desc = could not find container \"dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22\": container with ID starting with dd3f85ee5a0dc4251fbe515482a6afacf48741797c894476fdb660ff142bda22 not found: ID does not exist"
Sep 18 20:29:48.393483 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.393462    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:29:48.397087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:48.397051    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-a-77f5c1-openshift-default-5566bf7694-qx4g5"]
Sep 18 20:29:50.081560 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:50.081528    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" path="/var/lib/kubelet/pods/2f6b1f43-0a89-412c-8bb2-74c94b69a9a9/volumes"
Sep 18 20:29:58.661251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:58.661214    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:29:58.661842 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:58.661812    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" podUID="89e1fa92-fba8-4319-b525-8f040e858acc" containerName="payload-pre-processing" containerID="cri-o://371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94" gracePeriod=30
Sep 18 20:29:58.823580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:58.823522    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"]
Sep 18 20:29:58.826182 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:58.826159    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-b-102c83-ncxpq"]
Sep 18 20:29:58.949284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:58.949220    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:29:59.057933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.057900    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume\") pod \"89e1fa92-fba8-4319-b525-8f040e858acc\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") "
Sep 18 20:29:59.058120 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.058006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-545tg\" (UniqueName: \"kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg\") pod \"89e1fa92-fba8-4319-b525-8f040e858acc\" (UID: \"89e1fa92-fba8-4319-b525-8f040e858acc\") "
Sep 18 20:29:59.058292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.058269    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "89e1fa92-fba8-4319-b525-8f040e858acc" (UID: "89e1fa92-fba8-4319-b525-8f040e858acc"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:29:59.060095 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.060074    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg" (OuterVolumeSpecName: "kube-api-access-545tg") pod "89e1fa92-fba8-4319-b525-8f040e858acc" (UID: "89e1fa92-fba8-4319-b525-8f040e858acc"). InnerVolumeSpecName "kube-api-access-545tg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:29:59.159114 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.159079    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/89e1fa92-fba8-4319-b525-8f040e858acc-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.159114 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.159109    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-545tg\" (UniqueName: \"kubernetes.io/projected/89e1fa92-fba8-4319-b525-8f040e858acc-kube-api-access-545tg\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:29:59.435597 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.435561    2566 generic.go:356] "Generic (PLEG): container finished" podID="89e1fa92-fba8-4319-b525-8f040e858acc" containerID="371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94" exitCode=0
Sep 18 20:29:59.435773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.435608    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"
Sep 18 20:29:59.435773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.435647    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" event={"ID":"89e1fa92-fba8-4319-b525-8f040e858acc","Type":"ContainerDied","Data":"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"}
Sep 18 20:29:59.435773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.435705    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj" event={"ID":"89e1fa92-fba8-4319-b525-8f040e858acc","Type":"ContainerDied","Data":"4d889ffbe98c12e2e30baee952167fdb0173217eeee40d0bcd5b1ce663421d06"}
Sep 18 20:29:59.435773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.435734    2566 scope.go:117] "RemoveContainer" containerID="371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"
Sep 18 20:29:59.446550 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.446533    2566 scope.go:117] "RemoveContainer" containerID="371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"
Sep 18 20:29:59.446786 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:29:59.446769    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94\": container with ID starting with 371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94 not found: ID does not exist" containerID="371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"
Sep 18 20:29:59.446854 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.446796    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94"} err="failed to get container status \"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94\": rpc error: code = NotFound desc = could not find container \"371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94\": container with ID starting with 371fa430e1bb34ce7337ad312c66f429b92aac533b0a296840df0b7b047c9f94 not found: ID does not exist"
Sep 18 20:29:59.460102 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.460078    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:29:59.466082 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:29:59.466036    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-102c83-59d466f894-6xwbj"]
Sep 18 20:30:00.084496 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.084462    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89e1fa92-fba8-4319-b525-8f040e858acc" path="/var/lib/kubelet/pods/89e1fa92-fba8-4319-b525-8f040e858acc/volumes"
Sep 18 20:30:00.084864 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.084810    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4114aaf-276a-465c-aeab-53f70af7adc7" path="/var/lib/kubelet/pods/b4114aaf-276a-465c-aeab-53f70af7adc7/volumes"
Sep 18 20:30:00.131479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131445    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"]
Sep 18 20:30:00.131938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131921    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.131938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131938    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131953    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131959    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131978    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89e1fa92-fba8-4319-b525-8f040e858acc" containerName="payload-pre-processing"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131985    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="89e1fa92-fba8-4319-b525-8f040e858acc" containerName="payload-pre-processing"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.131997    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4114aaf-276a-465c-aeab-53f70af7adc7" containerName="revoke-keys"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.132004    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4114aaf-276a-465c-aeab-53f70af7adc7" containerName="revoke-keys"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.132099    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="89e1fa92-fba8-4319-b525-8f040e858acc" containerName="payload-pre-processing"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.132107    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.132117    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b4114aaf-276a-465c-aeab-53f70af7adc7" containerName="revoke-keys"
Sep 18 20:30:00.132173 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.132127    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2f6b1f43-0a89-412c-8bb2-74c94b69a9a9" containerName="istio-proxy"
Sep 18 20:30:00.136797 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.136777    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"]
Sep 18 20:30:00.136904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.136883    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"
Sep 18 20:30:00.139373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.139348    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:30:00.140205 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.140186    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"]
Sep 18 20:30:00.140337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.140283    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"
Sep 18 20:30:00.144318 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.144298    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"
Sep 18 20:30:00.144806 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.144784    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"]
Sep 18 20:30:00.148343 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.148322    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"]
Sep 18 20:30:00.151132 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.151115    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"
Sep 18 20:30:00.162536 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.162508    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"]
Sep 18 20:30:00.163717 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.163695    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"
Sep 18 20:30:00.207329 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.207169    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"
Sep 18 20:30:00.435032 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.435005    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"]
Sep 18 20:30:00.436691 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:00.436663    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc6c0d1b_bdef_41a6_87ae_f0fbf07ea1c6.slice/crio-821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605 WatchSource:0}: Error finding container 821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605: Status 404 returned error can't find the container with id 821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605
Sep 18 20:30:00.466507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.466477    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"]
Sep 18 20:30:00.471018 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:00.470979    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod005d4f8a_71db_44a4_9a6e_a15a22b7e5bd.slice/crio-6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2 WatchSource:0}: Error finding container 6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2: Status 404 returned error can't find the container with id 6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2
Sep 18 20:30:00.521351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:00.521326    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"]
Sep 18 20:30:00.522116 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:00.522095    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod45ccf214_aa02_4199_991b_d55e0e66240b.slice/crio-0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23 WatchSource:0}: Error finding container 0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23: Status 404 returned error can't find the container with id 0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23
Sep 18 20:30:01.461940 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.461909    2566 generic.go:356] "Generic (PLEG): container finished" podID="cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6" containerID="a7174dbef0b737f582b4d65af104421418824d637f2513a505cea6c7546f8bac" exitCode=0
Sep 18 20:30:01.462293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.461996    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk" event={"ID":"cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6","Type":"ContainerDied","Data":"a7174dbef0b737f582b4d65af104421418824d637f2513a505cea6c7546f8bac"}
Sep 18 20:30:01.462293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.462051    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk" event={"ID":"cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6","Type":"ContainerStarted","Data":"821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605"}
Sep 18 20:30:01.463507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.463479    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm" event={"ID":"005d4f8a-71db-44a4-9a6e-a15a22b7e5bd","Type":"ContainerStarted","Data":"9fc9925677f44ce338cd79d7857c3d2aa465d4103a61ae725edf36d8410ded49"}
Sep 18 20:30:01.463598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.463525    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm" event={"ID":"005d4f8a-71db-44a4-9a6e-a15a22b7e5bd","Type":"ContainerStarted","Data":"6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2"}
Sep 18 20:30:01.464956 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.464936    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s" event={"ID":"45ccf214-aa02-4199-991b-d55e0e66240b","Type":"ContainerStarted","Data":"828bc8a69922ecca728e6b8aa7e8385be826941d65e193a2a55031f337f02456"}
Sep 18 20:30:01.465095 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.464971    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s" event={"ID":"45ccf214-aa02-4199-991b-d55e0e66240b","Type":"ContainerStarted","Data":"0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23"}
Sep 18 20:30:01.496290 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:01.496241    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s" podStartSLOduration=1.496227177 podStartE2EDuration="1.496227177s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:01.494748749 +0000 UTC m=+1745.624050703" watchObservedRunningTime="2026-09-18 20:30:01.496227177 +0000 UTC m=+1745.625529128"
Sep 18 20:30:02.476999 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:02.476957    2566 generic.go:356] "Generic (PLEG): container finished" podID="005d4f8a-71db-44a4-9a6e-a15a22b7e5bd" containerID="9fc9925677f44ce338cd79d7857c3d2aa465d4103a61ae725edf36d8410ded49" exitCode=0
Sep 18 20:30:02.477545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:02.477441    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm" event={"ID":"005d4f8a-71db-44a4-9a6e-a15a22b7e5bd","Type":"ContainerDied","Data":"9fc9925677f44ce338cd79d7857c3d2aa465d4103a61ae725edf36d8410ded49"}
Sep 18 20:30:02.479849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:02.479824    2566 generic.go:356] "Generic (PLEG): container finished" podID="45ccf214-aa02-4199-991b-d55e0e66240b" containerID="828bc8a69922ecca728e6b8aa7e8385be826941d65e193a2a55031f337f02456" exitCode=0
Sep 18 20:30:02.480076 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:02.479885    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s" event={"ID":"45ccf214-aa02-4199-991b-d55e0e66240b","Type":"ContainerDied","Data":"828bc8a69922ecca728e6b8aa7e8385be826941d65e193a2a55031f337f02456"}
Sep 18 20:30:02.613195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:02.613172    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"
Sep 18 20:30:03.488632 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:03.488603    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"
Sep 18 20:30:03.489046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:03.488642    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk" event={"ID":"cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6","Type":"ContainerDied","Data":"821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605"}
Sep 18 20:30:03.489046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:03.488691    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="821a6d233b5803e3c75e91d68b6a86526870a8089ad4708bdf11630ddb2fc605"
Sep 18 20:30:03.658010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:03.657982    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"
Sep 18 20:30:03.660740 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:03.660720    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"
Sep 18 20:30:04.496032 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.496005    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"
Sep 18 20:30:04.496484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.496024    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm" event={"ID":"005d4f8a-71db-44a4-9a6e-a15a22b7e5bd","Type":"ContainerDied","Data":"6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2"}
Sep 18 20:30:04.496484 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.496133    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6175e9c97729258d758238c44e19457681ed188056ecb372a664bec835c200b2"
Sep 18 20:30:04.497473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.497451    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s" event={"ID":"45ccf214-aa02-4199-991b-d55e0e66240b","Type":"ContainerDied","Data":"0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23"}
Sep 18 20:30:04.497590 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.497531    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0c4eaea34253e7ddf5f9c1a91e2b6d565603a788f02073deacbdb64e06f63e23"
Sep 18 20:30:04.497590 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:04.497455    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-29829390-m2c4s"
Sep 18 20:30:05.352002 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.351970    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:30:05.352271 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.352212    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy" containerID="cri-o://2a55befb1d3728b1aa444d7275725b4525c72553be3f7da76192dfc347aee873" gracePeriod=30
Sep 18 20:30:05.686975 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.686899    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"]
Sep 18 20:30:05.687361 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687347    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="45ccf214-aa02-4199-991b-d55e0e66240b" containerName="cleanup"
Sep 18 20:30:05.687361 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687361    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="45ccf214-aa02-4199-991b-d55e0e66240b" containerName="cleanup"
Sep 18 20:30:05.687437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687373    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="005d4f8a-71db-44a4-9a6e-a15a22b7e5bd" containerName="cleanup"
Sep 18 20:30:05.687437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687378    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="005d4f8a-71db-44a4-9a6e-a15a22b7e5bd" containerName="cleanup"
Sep 18 20:30:05.687437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687400    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6" containerName="cleanup"
Sep 18 20:30:05.687437 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687405    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6" containerName="cleanup"
Sep 18 20:30:05.687558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687475    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="005d4f8a-71db-44a4-9a6e-a15a22b7e5bd" containerName="cleanup"
Sep 18 20:30:05.687558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687481    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6" containerName="cleanup"
Sep 18 20:30:05.687558 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.687486    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="45ccf214-aa02-4199-991b-d55e0e66240b" containerName="cleanup"
Sep 18 20:30:05.690727 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.690712    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:05.693193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.693169    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:30:05.693193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.693179    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:30:05.698295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.697975    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"]
Sep 18 20:30:05.826187 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.826157    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq\" (UID: \"1abc17d0-bba0-4fa8-bee1-a54c5311a280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:05.927365 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.927324    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq\" (UID: \"1abc17d0-bba0-4fa8-bee1-a54c5311a280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:05.927953 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:05.927936    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq\" (UID: \"1abc17d0-bba0-4fa8-bee1-a54c5311a280\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:06.002177 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:06.002151    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:06.278079 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:06.278049    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"]
Sep 18 20:30:06.280955 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:06.280928    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1abc17d0_bba0_4fa8_bee1_a54c5311a280.slice/crio-3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26 WatchSource:0}: Error finding container 3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26: Status 404 returned error can't find the container with id 3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26
Sep 18 20:30:06.510489 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:06.510460    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq" event={"ID":"1abc17d0-bba0-4fa8-bee1-a54c5311a280","Type":"ContainerStarted","Data":"3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26"}
Sep 18 20:30:07.518468 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:07.518428    2566 generic.go:356] "Generic (PLEG): container finished" podID="1abc17d0-bba0-4fa8-bee1-a54c5311a280" containerID="00a4b923895e2cc61b72f13968b40f3434a6d6b38cce3e44807f4227fbb21608" exitCode=0
Sep 18 20:30:07.518857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:07.518500    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq" event={"ID":"1abc17d0-bba0-4fa8-bee1-a54c5311a280","Type":"ContainerDied","Data":"00a4b923895e2cc61b72f13968b40f3434a6d6b38cce3e44807f4227fbb21608"}
Sep 18 20:30:08.661196 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:08.661171    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:08.756725 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:08.756692    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca\") pod \"1abc17d0-bba0-4fa8-bee1-a54c5311a280\" (UID: \"1abc17d0-bba0-4fa8-bee1-a54c5311a280\") "
Sep 18 20:30:08.757089 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:08.757045    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1abc17d0-bba0-4fa8-bee1-a54c5311a280" (UID: "1abc17d0-bba0-4fa8-bee1-a54c5311a280"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:08.857990 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:08.857903    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1abc17d0-bba0-4fa8-bee1-a54c5311a280-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:09.532486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:09.532444    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq" event={"ID":"1abc17d0-bba0-4fa8-bee1-a54c5311a280","Type":"ContainerDied","Data":"3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26"}
Sep 18 20:30:09.532486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:09.532467    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"
Sep 18 20:30:09.532747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:09.532508    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3f1a24b7c0e4f7506bb7e8291ea445b63ebe22b9fab26b977e860736f7367b26"
Sep 18 20:30:10.544384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.544344    2566 generic.go:356] "Generic (PLEG): container finished" podID="b1609fa2-bad7-48d7-a870-e7250f052786" containerID="2a55befb1d3728b1aa444d7275725b4525c72553be3f7da76192dfc347aee873" exitCode=0
Sep 18 20:30:10.544794 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.544395    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerDied","Data":"2a55befb1d3728b1aa444d7275725b4525c72553be3f7da76192dfc347aee873"}
Sep 18 20:30:10.544794 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.544479    2566 scope.go:117] "RemoveContainer" containerID="c79b197aaf4d7bd54ae1b0bfb1103d1377047a1962a81754eafa63225f555f8e"
Sep 18 20:30:10.619106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.619042    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:30:10.777266 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777232    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777266 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777270    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777318    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777344    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777363    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777405    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777458    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gxxd4\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777501    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777545    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:10.777812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777558    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket\") pod \"b1609fa2-bad7-48d7-a870-e7250f052786\" (UID: \"b1609fa2-bad7-48d7-a870-e7250f052786\") "
Sep 18 20:30:10.777812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777601    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:10.777812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777770    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data" (OuterVolumeSpecName: "istio-data") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:10.777812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.777804    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:10.778106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.778043    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:10.778177 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.778131    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.778177 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.778150    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.778177 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.778165    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.778337 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.778178    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.779718 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.779693    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:10.779814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.779732    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:30:10.779814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.779739    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token" (OuterVolumeSpecName: "istio-token") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:10.779814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.779748    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4" (OuterVolumeSpecName: "kube-api-access-gxxd4") pod "b1609fa2-bad7-48d7-a870-e7250f052786" (UID: "b1609fa2-bad7-48d7-a870-e7250f052786"). InnerVolumeSpecName "kube-api-access-gxxd4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:10.879314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.879238    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b1609fa2-bad7-48d7-a870-e7250f052786-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.879314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.879264    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b1609fa2-bad7-48d7-a870-e7250f052786-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.879314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.879272    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.879314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.879280    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gxxd4\" (UniqueName: \"kubernetes.io/projected/b1609fa2-bad7-48d7-a870-e7250f052786-kube-api-access-gxxd4\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:10.879314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:10.879291    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b1609fa2-bad7-48d7-a870-e7250f052786-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:11.552163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:11.552131    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"
Sep 18 20:30:11.552625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:11.552172    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf" event={"ID":"b1609fa2-bad7-48d7-a870-e7250f052786","Type":"ContainerDied","Data":"d1b1e54d3577af8155de3f10cbff72132fd07bb0d7b7000d0b997bb2edc520ab"}
Sep 18 20:30:11.552625 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:11.552243    2566 scope.go:117] "RemoveContainer" containerID="2a55befb1d3728b1aa444d7275725b4525c72553be3f7da76192dfc347aee873"
Sep 18 20:30:11.574884 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:11.574861    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:30:11.580667 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:11.580636    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-api-b-102c83-openshift-default-54c6c7d5b4-ltmbf"]
Sep 18 20:30:12.081550 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:12.081521    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" path="/var/lib/kubelet/pods/b1609fa2-bad7-48d7-a870-e7250f052786/volumes"
Sep 18 20:30:16.238340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.238303    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:30:16.238816 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.238556    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" podUID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" containerName="payload-processing" containerID="cri-o://4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c" gracePeriod=30
Sep 18 20:30:16.253323 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.253294    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:30:16.253549 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.253523    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerName="payload-pre-processing" containerID="cri-o://aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9" gracePeriod=30
Sep 18 20:30:16.418416 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.418277    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"]
Sep 18 20:30:16.422210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.422174    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w1-35f893-pjlzq"]
Sep 18 20:30:16.509581 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.509514    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:30:16.532450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.532329    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume\") pod \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") "
Sep 18 20:30:16.532450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.532392    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xqfp2\" (UniqueName: \"kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2\") pod \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\" (UID: \"4356f317-a2b8-47e4-a7ff-b1d8b953467a\") "
Sep 18 20:30:16.532862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.532820    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "4356f317-a2b8-47e4-a7ff-b1d8b953467a" (UID: "4356f317-a2b8-47e4-a7ff-b1d8b953467a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:16.535819 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.535790    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2" (OuterVolumeSpecName: "kube-api-access-xqfp2") pod "4356f317-a2b8-47e4-a7ff-b1d8b953467a" (UID: "4356f317-a2b8-47e4-a7ff-b1d8b953467a"). InnerVolumeSpecName "kube-api-access-xqfp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:16.549455 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.549433    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.592345    2566 generic.go:356] "Generic (PLEG): container finished" podID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerID="aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9" exitCode=0
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.592370    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" event={"ID":"75971fd6-a93f-4958-ba8b-ea475b3cc36a","Type":"ContainerDied","Data":"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"}
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.592426    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" event={"ID":"75971fd6-a93f-4958-ba8b-ea475b3cc36a","Type":"ContainerDied","Data":"0295a81bd4f79ef15c02ab0631ab37bda3fac43271808e63df8061c88c446a24"}
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.592456    2566 scope.go:117] "RemoveContainer" containerID="aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.592539    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.594869    2566 generic.go:356] "Generic (PLEG): container finished" podID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" containerID="4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c" exitCode=0
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.595043    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.595244    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" event={"ID":"4356f317-a2b8-47e4-a7ff-b1d8b953467a","Type":"ContainerDied","Data":"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"}
Sep 18 20:30:16.599083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.595286    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t" event={"ID":"4356f317-a2b8-47e4-a7ff-b1d8b953467a","Type":"ContainerDied","Data":"e5fd3e4c06f17e3a24aa9ef9247f8b5bb0f033595a33c7bd7be90c88fe0e0682"}
Sep 18 20:30:16.616531 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.616507    2566 scope.go:117] "RemoveContainer" containerID="aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"
Sep 18 20:30:16.617186 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:16.617165    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9\": container with ID starting with aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9 not found: ID does not exist" containerID="aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"
Sep 18 20:30:16.617293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.617195    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9"} err="failed to get container status \"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9\": rpc error: code = NotFound desc = could not find container \"aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9\": container with ID starting with aa97e041d40b8849e4b766d2d04b38a48d40d993ef2a10ffe1467f6450322bc9 not found: ID does not exist"
Sep 18 20:30:16.617293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.617211    2566 scope.go:117] "RemoveContainer" containerID="4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"
Sep 18 20:30:16.626909 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.626885    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:30:16.627248 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.627235    2566 scope.go:117] "RemoveContainer" containerID="4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"
Sep 18 20:30:16.627537 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:16.627513    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c\": container with ID starting with 4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c not found: ID does not exist" containerID="4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"
Sep 18 20:30:16.627638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.627541    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c"} err="failed to get container status \"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c\": rpc error: code = NotFound desc = could not find container \"4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c\": container with ID starting with 4cf7781de31869f83adfe2651a79e0c593c98c6d1ff2b440106301bd9b10ae7c not found: ID does not exist"
Sep 18 20:30:16.630422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.630399    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w1-35f893-6c7597959-x2z2t"]
Sep 18 20:30:16.633353 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633327    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume\") pod \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") "
Sep 18 20:30:16.633475 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633419    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f74pq\" (UniqueName: \"kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq\") pod \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\" (UID: \"75971fd6-a93f-4958-ba8b-ea475b3cc36a\") "
Sep 18 20:30:16.633724 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633701    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "75971fd6-a93f-4958-ba8b-ea475b3cc36a" (UID: "75971fd6-a93f-4958-ba8b-ea475b3cc36a"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:16.633833 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633817    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4356f317-a2b8-47e4-a7ff-b1d8b953467a-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:16.633895 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633835    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xqfp2\" (UniqueName: \"kubernetes.io/projected/4356f317-a2b8-47e4-a7ff-b1d8b953467a-kube-api-access-xqfp2\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:16.633895 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.633844    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/75971fd6-a93f-4958-ba8b-ea475b3cc36a-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:16.635370 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.635343    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq" (OuterVolumeSpecName: "kube-api-access-f74pq") pod "75971fd6-a93f-4958-ba8b-ea475b3cc36a" (UID: "75971fd6-a93f-4958-ba8b-ea475b3cc36a"). InnerVolumeSpecName "kube-api-access-f74pq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:16.735456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.735417    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f74pq\" (UniqueName: \"kubernetes.io/projected/75971fd6-a93f-4958-ba8b-ea475b3cc36a-kube-api-access-f74pq\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:16.913873 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.913843    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:30:16.919312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:16.919285    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh"]
Sep 18 20:30:17.534666 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:17.534577    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w1-35f893-777f847cc8-5clxh" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerName="payload-pre-processing" probeResult="failure" output="dial tcp 10.132.0.73:9004: i/o timeout"
Sep 18 20:30:18.083609 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:18.083567    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1abc17d0-bba0-4fa8-bee1-a54c5311a280" path="/var/lib/kubelet/pods/1abc17d0-bba0-4fa8-bee1-a54c5311a280/volumes"
Sep 18 20:30:18.084138 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:18.084117    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" path="/var/lib/kubelet/pods/4356f317-a2b8-47e4-a7ff-b1d8b953467a/volumes"
Sep 18 20:30:18.084638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:18.084619    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" path="/var/lib/kubelet/pods/75971fd6-a93f-4958-ba8b-ea475b3cc36a/volumes"
Sep 18 20:30:22.718292 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:22.718255    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:30:22.718703 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:22.718530    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy" containerID="cri-o://b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543" gracePeriod=30
Sep 18 20:30:23.051514 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.051419    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"]
Sep 18 20:30:23.052490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052460    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:23.052490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052490    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052520    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1abc17d0-bba0-4fa8-bee1-a54c5311a280" containerName="revoke-keys"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052530    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1abc17d0-bba0-4fa8-bee1-a54c5311a280" containerName="revoke-keys"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052568    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerName="payload-pre-processing"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052577    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerName="payload-pre-processing"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052613    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" containerName="payload-processing"
Sep 18 20:30:23.052676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052629    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" containerName="payload-processing"
Sep 18 20:30:23.052941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052808    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1abc17d0-bba0-4fa8-bee1-a54c5311a280" containerName="revoke-keys"
Sep 18 20:30:23.052941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052824    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:23.052941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052842    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:23.052941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052851    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="75971fd6-a93f-4958-ba8b-ea475b3cc36a" containerName="payload-pre-processing"
Sep 18 20:30:23.052941 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.052862    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="4356f317-a2b8-47e4-a7ff-b1d8b953467a" containerName="payload-processing"
Sep 18 20:30:23.057573 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.057545    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:23.060044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.060016    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"]
Sep 18 20:30:23.060275 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.060251    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:30:23.060425 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.060408    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:30:23.093775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.093747    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb\" (UID: \"0c85cfc4-282f-443d-b53a-aff0aa26746d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:23.195456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.195429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb\" (UID: \"0c85cfc4-282f-443d-b53a-aff0aa26746d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:23.196009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.195990    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb\" (UID: \"0c85cfc4-282f-443d-b53a-aff0aa26746d\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:23.370785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.370718    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:23.644843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:23.644817    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"]
Sep 18 20:30:23.647905 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:23.647877    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c85cfc4_282f_443d_b53a_aff0aa26746d.slice/crio-fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a WatchSource:0}: Error finding container fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a: Status 404 returned error can't find the container with id fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a
Sep 18 20:30:24.648571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:24.648487    2566 generic.go:356] "Generic (PLEG): container finished" podID="0c85cfc4-282f-443d-b53a-aff0aa26746d" containerID="cd7da44b8ea5888ffd1498e95d7cc9ad44e363bda19b11b70d15a70621937d8d" exitCode=0
Sep 18 20:30:24.648919 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:24.648579    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb" event={"ID":"0c85cfc4-282f-443d-b53a-aff0aa26746d","Type":"ContainerDied","Data":"cd7da44b8ea5888ffd1498e95d7cc9ad44e363bda19b11b70d15a70621937d8d"}
Sep 18 20:30:24.648919 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:24.648649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb" event={"ID":"0c85cfc4-282f-443d-b53a-aff0aa26746d","Type":"ContainerStarted","Data":"fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a"}
Sep 18 20:30:25.791467 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:25.791444    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:25.818557 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:25.818522    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca\") pod \"0c85cfc4-282f-443d-b53a-aff0aa26746d\" (UID: \"0c85cfc4-282f-443d-b53a-aff0aa26746d\") "
Sep 18 20:30:25.819247 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:25.819220    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "0c85cfc4-282f-443d-b53a-aff0aa26746d" (UID: "0c85cfc4-282f-443d-b53a-aff0aa26746d"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:25.919526 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:25.919450    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/0c85cfc4-282f-443d-b53a-aff0aa26746d-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:26.661615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:26.661583    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb" event={"ID":"0c85cfc4-282f-443d-b53a-aff0aa26746d","Type":"ContainerDied","Data":"fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a"}
Sep 18 20:30:26.661758 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:26.661618    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"
Sep 18 20:30:26.661758 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:26.661630    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="fbf57faad1c6f713e9eb4645b1205bf8127a48e606da1d77c87b72ef100a796a"
Sep 18 20:30:27.971491 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:27.971470    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:30:28.037414 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037341    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037417    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037451    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p4sbv\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037488    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037551 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037539    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037579    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037608    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037689    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037726    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037732    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token\") pod \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\" (UID: \"9d97ba05-1dcb-4995-a4c0-0b8c8981c062\") "
Sep 18 20:30:28.037855 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037814    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:28.038265 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037918    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:28.038265 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.037952    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data" (OuterVolumeSpecName: "istio-data") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:28.038265 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038180    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:28.038430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038265    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.038430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038280    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.038430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038289    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.038430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038299    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.038430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.038307    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.039661 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.039636    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:28.039874 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.039853    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token" (OuterVolumeSpecName: "istio-token") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:28.039967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.039891    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv" (OuterVolumeSpecName: "kube-api-access-p4sbv") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "kube-api-access-p4sbv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:28.040088 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.040039    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "9d97ba05-1dcb-4995-a4c0-0b8c8981c062" (UID: "9d97ba05-1dcb-4995-a4c0-0b8c8981c062"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:30:28.138987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.138957    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.138987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.138984    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.138987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.138994    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p4sbv\" (UniqueName: \"kubernetes.io/projected/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-kube-api-access-p4sbv\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.139277 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.139003    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/9d97ba05-1dcb-4995-a4c0-0b8c8981c062-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:28.675850 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.675813    2566 generic.go:356] "Generic (PLEG): container finished" podID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerID="b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543" exitCode=0
Sep 18 20:30:28.676031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.675888    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"
Sep 18 20:30:28.676031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.675889    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerDied","Data":"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"}
Sep 18 20:30:28.676031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.676006    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf" event={"ID":"9d97ba05-1dcb-4995-a4c0-0b8c8981c062","Type":"ContainerDied","Data":"68d5107338becf51191e761c64d7c0bb622e88dafd0c350d2bf742497e06cb21"}
Sep 18 20:30:28.676031 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.676030    2566 scope.go:117] "RemoveContainer" containerID="b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"
Sep 18 20:30:28.685717 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.685687    2566 scope.go:117] "RemoveContainer" containerID="1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"
Sep 18 20:30:28.693892 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.693867    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:30:28.697398 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.697378    2566 scope.go:117] "RemoveContainer" containerID="b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"
Sep 18 20:30:28.697914 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:28.697892    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543\": container with ID starting with b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543 not found: ID does not exist" containerID="b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"
Sep 18 20:30:28.698010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.697924    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543"} err="failed to get container status \"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543\": rpc error: code = NotFound desc = could not find container \"b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543\": container with ID starting with b05938c983855cbe4999a3c707c6b4bf8b0849615ec6f460e7fb2229555ac543 not found: ID does not exist"
Sep 18 20:30:28.698010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.697944    2566 scope.go:117] "RemoveContainer" containerID="1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"
Sep 18 20:30:28.698494 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:28.698474    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd\": container with ID starting with 1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd not found: ID does not exist" containerID="1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"
Sep 18 20:30:28.698584 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.698497    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd"} err="failed to get container status \"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd\": rpc error: code = NotFound desc = could not find container \"1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd\": container with ID starting with 1e932c75ee37d992a985b8bdbe77addcd2dd70a5b922057fb7b06070048569dd not found: ID does not exist"
Sep 18 20:30:28.698640 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:28.698623    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w1-35f893-openshift-default-585dfd9fd4-zmzbf"]
Sep 18 20:30:30.082716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:30.082681    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" path="/var/lib/kubelet/pods/9d97ba05-1dcb-4995-a4c0-0b8c8981c062/volumes"
Sep 18 20:30:33.642841 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:33.642812    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"]
Sep 18 20:30:33.647879 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:33.647854    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-b-w1-9a5190-29829390-xxgdm"]
Sep 18 20:30:33.918147 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:33.918050    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"]
Sep 18 20:30:33.925932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:33.922547    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w1-9a5190-npzfb"]
Sep 18 20:30:34.082170 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:34.082133    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="005d4f8a-71db-44a4-9a6e-a15a22b7e5bd" path="/var/lib/kubelet/pods/005d4f8a-71db-44a4-9a6e-a15a22b7e5bd/volumes"
Sep 18 20:30:34.082656 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:34.082639    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c85cfc4-282f-443d-b53a-aff0aa26746d" path="/var/lib/kubelet/pods/0c85cfc4-282f-443d-b53a-aff0aa26746d/volumes"
Sep 18 20:30:40.494555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:40.494509    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:30:40.495050 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:40.494735    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" containerID="cri-o://f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3" gracePeriod=30
Sep 18 20:30:42.116385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:42.116354    2566 patch_prober.go:28] interesting pod/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 18 20:30:42.116761 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:42.116409    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 18 20:30:45.748486 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.748463    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:30:45.789328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.789234    2566 generic.go:356] "Generic (PLEG): container finished" podID="b978ea71-2c96-4928-a4fb-560cec27d317" containerID="f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3" exitCode=0
Sep 18 20:30:45.789328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.789314    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"
Sep 18 20:30:45.789509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.789345    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerDied","Data":"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"}
Sep 18 20:30:45.789509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.789423    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2" event={"ID":"b978ea71-2c96-4928-a4fb-560cec27d317","Type":"ContainerDied","Data":"1e19ccf3293c1897c0233a3079689a60bd7b9b34a1ee3534f38e08e8ffcdb42d"}
Sep 18 20:30:45.789509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.789455    2566 scope.go:117] "RemoveContainer" containerID="f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"
Sep 18 20:30:45.799459 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.799436    2566 scope.go:117] "RemoveContainer" containerID="babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"
Sep 18 20:30:45.807954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.807938    2566 scope.go:117] "RemoveContainer" containerID="f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"
Sep 18 20:30:45.808253 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:45.808235    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3\": container with ID starting with f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3 not found: ID does not exist" containerID="f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"
Sep 18 20:30:45.808320 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.808261    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3"} err="failed to get container status \"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3\": rpc error: code = NotFound desc = could not find container \"f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3\": container with ID starting with f89cbcdd63fbe5a5c1643de9518d5fd59ef273ba73c89952cf783aadde3222d3 not found: ID does not exist"
Sep 18 20:30:45.808320 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.808278    2566 scope.go:117] "RemoveContainer" containerID="babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"
Sep 18 20:30:45.808529 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:30:45.808512    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472\": container with ID starting with babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472 not found: ID does not exist" containerID="babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"
Sep 18 20:30:45.808571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.808535    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472"} err="failed to get container status \"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472\": rpc error: code = NotFound desc = could not find container \"babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472\": container with ID starting with babc53ae892c28b5edbce377144a62aa1e902cc4735bdf2ebb23b3ca9b3a9472 not found: ID does not exist"
Sep 18 20:30:45.900391 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900360    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900403    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900448    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900474    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900531    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900588    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900607    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4vfrn\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900621    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900647    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data\") pod \"b978ea71-2c96-4928-a4fb-560cec27d317\" (UID: \"b978ea71-2c96-4928-a4fb-560cec27d317\") "
Sep 18 20:30:45.900943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900761    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:45.900943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900859    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:45.900943 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900923    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:45.901116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.900969    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:30:45.901116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.901051    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:45.901116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.901096    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:45.901116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.901109    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:45.901325 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.901166    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data" (OuterVolumeSpecName: "istio-data") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:45.902639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.902612    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token" (OuterVolumeSpecName: "istio-token") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:45.902781 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.902764    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:30:45.902840 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.902824    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:30:45.902895 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:45.902867    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn" (OuterVolumeSpecName: "kube-api-access-4vfrn") pod "b978ea71-2c96-4928-a4fb-560cec27d317" (UID: "b978ea71-2c96-4928-a4fb-560cec27d317"). InnerVolumeSpecName "kube-api-access-4vfrn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:30:46.002479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002438    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4vfrn\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-kube-api-access-4vfrn\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.002479 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002479    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.002750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002493    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b978ea71-2c96-4928-a4fb-560cec27d317-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.002750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002505    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b978ea71-2c96-4928-a4fb-560cec27d317-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.002750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002516    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b978ea71-2c96-4928-a4fb-560cec27d317-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.002750 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.002527    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b978ea71-2c96-4928-a4fb-560cec27d317-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:30:46.107364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.107334    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:30:46.109717 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:46.109692    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-b-w1-9a5190-openshift-default-786b5cc8dd-5lns2"]
Sep 18 20:30:48.080870 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:48.080840    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" path="/var/lib/kubelet/pods/b978ea71-2c96-4928-a4fb-560cec27d317/volumes"
Sep 18 20:30:53.402825 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.402793    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:30:53.403406 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403385    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403408    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403426    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403433    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403445    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c85cfc4-282f-443d-b53a-aff0aa26746d" containerName="revoke-keys"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403454    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c85cfc4-282f-443d-b53a-aff0aa26746d" containerName="revoke-keys"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403487    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.403499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403495    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403507    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403515    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1609fa2-bad7-48d7-a870-e7250f052786" containerName="istio-proxy"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403607    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403622    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="0c85cfc4-282f-443d-b53a-aff0aa26746d" containerName="revoke-keys"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403636    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.403828 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.403645    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b978ea71-2c96-4928-a4fb-560cec27d317" containerName="istio-proxy"
Sep 18 20:30:53.406900 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.406880    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.409430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.409405    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"kube-root-ca.crt\""
Sep 18 20:30:53.409430 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.409424    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"openshift-service-ca.crt\""
Sep 18 20:30:53.410453 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.410435    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"default-dockercfg-tppw2\""
Sep 18 20:30:53.410522 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.410482    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w2-bab50d\"/\"rate-test-model-bab50d-kserve-self-signed-certs\""
Sep 18 20:30:53.414686 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.414658    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:30:53.573112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573076    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.573322 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573164    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwbmc\" (UniqueName: \"kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.573399 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573311    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.573399 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573345    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.573512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573399    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.573512 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.573448    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674387    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674454    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwbmc\" (UniqueName: \"kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674565    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674591    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674695 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674680    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.674994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.674972    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.675083 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.675031    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.675151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.675073    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.677151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.677125    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.677402 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.677384    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.682324 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.682301    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwbmc\" (UniqueName: \"kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc\") pod \"rate-test-model-bab50d-kserve-7b84d48b87-bktwb\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") " pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.717585 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.717563    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:30:53.993575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:53.993545    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:30:53.995386 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:30:53.995356    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podabea1870_66c0_4e0c_adb3_887fb0ead557.slice/crio-09e4e8b2d5b02653d6292cd85e961d0e49e9ed2e9a138615ea1133113fbca1e5 WatchSource:0}: Error finding container 09e4e8b2d5b02653d6292cd85e961d0e49e9ed2e9a138615ea1133113fbca1e5: Status 404 returned error can't find the container with id 09e4e8b2d5b02653d6292cd85e961d0e49e9ed2e9a138615ea1133113fbca1e5
Sep 18 20:30:54.844023 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:54.843989    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" event={"ID":"abea1870-66c0-4e0c-adb3-887fb0ead557","Type":"ContainerStarted","Data":"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"}
Sep 18 20:30:54.844023 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:54.844028    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" event={"ID":"abea1870-66c0-4e0c-adb3-887fb0ead557","Type":"ContainerStarted","Data":"09e4e8b2d5b02653d6292cd85e961d0e49e9ed2e9a138615ea1133113fbca1e5"}
Sep 18 20:30:54.860930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:54.860885    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" podStartSLOduration=1.8608700790000001 podStartE2EDuration="1.860870079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:30:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:30:54.859434547 +0000 UTC m=+1798.988736499" watchObservedRunningTime="2026-09-18 20:30:54.860870079 +0000 UTC m=+1798.990172028"
Sep 18 20:30:58.195490 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:58.195459    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:30:58.199215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:58.199192    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:30:58.441697 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:30:58.441661    2566 scope.go:117] "RemoveContainer" containerID="5a593212543ec90fdefe87c7ddab8eb257ad4ded6d486932487f5c829468a7f5"
Sep 18 20:31:03.718582 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:31:03.718550    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:31:03.719217 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:31:03.718597    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:31:03.730820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:31:03.730799    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:31:03.904543 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:31:03.904516    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:32:13.266116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:13.266079    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:32:14.804038 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.804008    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:32:14.804631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.804610    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:32:14.804631 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.804631    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:32:14.804773 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.804713    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9d97ba05-1dcb-4995-a4c0-0b8c8981c062" containerName="istio-proxy"
Sep 18 20:32:14.807881 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.807858    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.810301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.810285    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-8cdaea-openshift-default-dockercfg-69b56\""
Sep 18 20:32:14.818860 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.818839    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:32:14.877669 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877643    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877768 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877685    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877768 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877706    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877849 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877815    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877900 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dn947\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877936 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877912    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877935    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.877980 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.877958    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.878116 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.878036    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978666    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dn947\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978699 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978705    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978729    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978745    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978836    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978905    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.978938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.978939    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979289 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979043    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979289 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979179    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979289 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979192    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979381    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979473 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979455    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.979785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.979764    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.981310 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.981288    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.981395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.981345    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.986188 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.986169    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dn947\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:14.986279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:14.986171    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token\") pod \"e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") " pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:15.118639 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:15.118579    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:15.393195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:15.393166    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:32:15.394933 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:32:15.394905    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6209c3ad_0097_430e_84ff_3d22a9542b25.slice/crio-7a125d53edb21425be07adeb03fadc557797a229328781d1728ffdbfce9f09ca WatchSource:0}: Error finding container 7a125d53edb21425be07adeb03fadc557797a229328781d1728ffdbfce9f09ca: Status 404 returned error can't find the container with id 7a125d53edb21425be07adeb03fadc557797a229328781d1728ffdbfce9f09ca
Sep 18 20:32:15.397144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:15.397116    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:15.397214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:15.397176    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:15.397214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:15.397202    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:16.300269 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:16.300225    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerStarted","Data":"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"}
Sep 18 20:32:16.300633 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:16.300285    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerStarted","Data":"7a125d53edb21425be07adeb03fadc557797a229328781d1728ffdbfce9f09ca"}
Sep 18 20:32:16.320446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:16.320403    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podStartSLOduration=2.320388185 podStartE2EDuration="2.320388185s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:14 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:16.318919088 +0000 UTC m=+1880.448221032" watchObservedRunningTime="2026-09-18 20:32:16.320388185 +0000 UTC m=+1880.449690134"
Sep 18 20:32:17.119405 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:17.119369    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:17.124111 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:17.124087    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:17.306959 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:17.306929    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:17.307958 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:17.307940    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:20.127982 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.127948    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:32:20.131601 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.131581    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.134100 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.134022    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-8cdaea-dockercfg-jhltj\""
Sep 18 20:32:20.134331 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.134313    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-8cdaea\""
Sep 18 20:32:20.140234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.140112    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:32:20.229814 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.229786    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.229933 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.229841    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-phrhn\" (UniqueName: \"kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.330868 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.330834    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.331044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.330886    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-phrhn\" (UniqueName: \"kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.331530 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.331509    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.339047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.339023    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-phrhn\" (UniqueName: \"kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn\") pod \"payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.444458 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.444398    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:20.714338 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:20.714316    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:32:20.716595 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:32:20.716568    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c902c18_15ff_410e_b6a0_dc9ef9d2fd98.slice/crio-c1dd8920b46df632a8323ee620366f861cabdac54823f70e9e3b801b4441165e WatchSource:0}: Error finding container c1dd8920b46df632a8323ee620366f861cabdac54823f70e9e3b801b4441165e: Status 404 returned error can't find the container with id c1dd8920b46df632a8323ee620366f861cabdac54823f70e9e3b801b4441165e
Sep 18 20:32:21.337450 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:21.337415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" event={"ID":"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98","Type":"ContainerStarted","Data":"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"}
Sep 18 20:32:21.337826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:21.337464    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" event={"ID":"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98","Type":"ContainerStarted","Data":"c1dd8920b46df632a8323ee620366f861cabdac54823f70e9e3b801b4441165e"}
Sep 18 20:32:21.337826 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:21.337495    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:21.355117 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:21.355071    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" podStartSLOduration=1.3550414370000001 podStartE2EDuration="1.355041437s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:21.35356536 +0000 UTC m=+1885.482867312" watchObservedRunningTime="2026-09-18 20:32:21.355041437 +0000 UTC m=+1885.484343386"
Sep 18 20:32:28.286283 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.286236    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" podUID="abea1870-66c0-4e0c-adb3-887fb0ead557" containerName="main" containerID="cri-o://c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9" gracePeriod=45
Sep 18 20:32:28.559498 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.559130    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:32:28.712731 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712701    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.712890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712754    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.712890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712789    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.712890 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712818    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qwbmc\" (UniqueName: \"kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.713048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712903    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.713048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712935    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm\") pod \"abea1870-66c0-4e0c-adb3-887fb0ead557\" (UID: \"abea1870-66c0-4e0c-adb3-887fb0ead557\") "
Sep 18 20:32:28.713048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.712936    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache" (OuterVolumeSpecName: "model-cache") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:28.713048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.713010    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:28.713048 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.713043    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home" (OuterVolumeSpecName: "home") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:28.713307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.713261    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-model-cache\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:28.713307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.713273    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-tmp-dir\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:28.713307 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.713282    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-home\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:28.715004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.714982    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm" (OuterVolumeSpecName: "dshm") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:32:28.715297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.715280    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:32:28.715367 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.715330    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc" (OuterVolumeSpecName: "kube-api-access-qwbmc") pod "abea1870-66c0-4e0c-adb3-887fb0ead557" (UID: "abea1870-66c0-4e0c-adb3-887fb0ead557"). InnerVolumeSpecName "kube-api-access-qwbmc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:32:28.814472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.814400    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/abea1870-66c0-4e0c-adb3-887fb0ead557-tls-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:28.814472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.814424    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/abea1870-66c0-4e0c-adb3-887fb0ead557-dshm\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:28.814472 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:28.814433    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qwbmc\" (UniqueName: \"kubernetes.io/projected/abea1870-66c0-4e0c-adb3-887fb0ead557-kube-api-access-qwbmc\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:32:29.390264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.390225    2566 generic.go:356] "Generic (PLEG): container finished" podID="abea1870-66c0-4e0c-adb3-887fb0ead557" containerID="c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9" exitCode=0
Sep 18 20:32:29.390264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.390263    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"
Sep 18 20:32:29.390783 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.390302    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" event={"ID":"abea1870-66c0-4e0c-adb3-887fb0ead557","Type":"ContainerDied","Data":"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"}
Sep 18 20:32:29.390783 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.390351    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb" event={"ID":"abea1870-66c0-4e0c-adb3-887fb0ead557","Type":"ContainerDied","Data":"09e4e8b2d5b02653d6292cd85e961d0e49e9ed2e9a138615ea1133113fbca1e5"}
Sep 18 20:32:29.390783 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.390370    2566 scope.go:117] "RemoveContainer" containerID="c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"
Sep 18 20:32:29.401870 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.401847    2566 scope.go:117] "RemoveContainer" containerID="c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"
Sep 18 20:32:29.402270 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:32:29.402252    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9\": container with ID starting with c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9 not found: ID does not exist" containerID="c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"
Sep 18 20:32:29.402347 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.402282    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9"} err="failed to get container status \"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9\": rpc error: code = NotFound desc = could not find container \"c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9\": container with ID starting with c9ed5b8320bab8b3ad6f2ad2490e5f29ed853f00782349bec71d0ec454a393e9 not found: ID does not exist"
Sep 18 20:32:29.414272 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.414245    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:32:29.417843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:29.417820    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w2-bab50d/rate-test-model-bab50d-kserve-7b84d48b87-bktwb"]
Sep 18 20:32:30.083218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:30.083184    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="abea1870-66c0-4e0c-adb3-887fb0ead557" path="/var/lib/kubelet/pods/abea1870-66c0-4e0c-adb3-887fb0ead557/volumes"
Sep 18 20:32:32.336747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.336716    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": read tcp 10.132.0.2:54586->10.132.0.95:15021: read: connection reset by peer" start-of-body=
Sep 18 20:32:32.337166 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.336820    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": read tcp 10.132.0.2:54586->10.132.0.95:15021: read: connection reset by peer"
Sep 18 20:32:32.346206 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.346181    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:32:32.411831 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.411801    2566 generic.go:356] "Generic (PLEG): container finished" podID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerID="db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789" exitCode=0
Sep 18 20:32:32.411978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.411876    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerDied","Data":"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"}
Sep 18 20:32:32.412298 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.412283    2566 scope.go:117] "RemoveContainer" containerID="db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"
Sep 18 20:32:32.413462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.413435    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:32.413560 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.413515    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:32.413621 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:32.413559    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:33.419994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:33.419959    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerStarted","Data":"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"}
Sep 18 20:32:34.119520 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:34.119487    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:34.119726 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:34.119694    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 18 20:32:34.119841 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:34.119751    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 18 20:32:35.119382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:35.119336    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 18 20:32:35.119790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:35.119403    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 18 20:32:36.120000 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:36.119962    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 18 20:32:36.120485 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:36.120020    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 18 20:32:37.118969 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:37.118930    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 18 20:32:37.119204 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:37.118991    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 18 20:32:38.119163 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:38.119122    2566 patch_prober.go:28] interesting pod/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused" start-of-body=
Sep 18 20:32:38.119529 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:38.119179    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.95:15021/healthz/ready\": dial tcp 10.132.0.95:15021: connect: connection refused"
Sep 18 20:32:39.122800 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:39.122769    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:39.123195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:39.123088    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:39.123766 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:39.123750    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:32:44.023446 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.023411    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:32:44.024105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.024067    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="abea1870-66c0-4e0c-adb3-887fb0ead557" containerName="main"
Sep 18 20:32:44.024105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.024088    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="abea1870-66c0-4e0c-adb3-887fb0ead557" containerName="main"
Sep 18 20:32:44.024251 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.024166    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="abea1870-66c0-4e0c-adb3-887fb0ead557" containerName="main"
Sep 18 20:32:44.026509 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.026490    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.029092 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.029048    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-667e4c-openshift-default-dockercfg-w5vr9\""
Sep 18 20:32:44.038754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.038727    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:32:44.063243 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063216    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063247    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063276    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063375 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063337    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063401    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063509    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063553    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063590    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.063649 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.063622    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8txjl\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.164803 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164770    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.164966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164807    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.164966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164850    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.164966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164884    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.164966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164916    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.164988    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165015    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165049    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165175 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165099    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8txjl\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165298 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165210    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165298 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165245    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165555 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165533    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165633 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165594    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.165766 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.165748    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.167393 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.167370    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.167730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.167708    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.172696 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.172667    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.172916 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.172897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8txjl\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl\") pod \"e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") " pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.337638 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.337495    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:44.643270 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.643233    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:32:44.644792 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:32:44.644765    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2c20e633_450a_4045_b617_020dbcebbd3b.slice/crio-90a0fb7c564f8070c54ea6f90a3a6aa4decb7a6f2796290e6896bc36cf3e482a WatchSource:0}: Error finding container 90a0fb7c564f8070c54ea6f90a3a6aa4decb7a6f2796290e6896bc36cf3e482a: Status 404 returned error can't find the container with id 90a0fb7c564f8070c54ea6f90a3a6aa4decb7a6f2796290e6896bc36cf3e482a
Sep 18 20:32:44.647746 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.647705    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:44.647824 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.647796    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:44.647870 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:44.647856    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:32:45.502327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:45.502294    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerStarted","Data":"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"}
Sep 18 20:32:45.502685 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:45.502346    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerStarted","Data":"90a0fb7c564f8070c54ea6f90a3a6aa4decb7a6f2796290e6896bc36cf3e482a"}
Sep 18 20:32:45.521263 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:45.521224    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podStartSLOduration=1.521207535 podStartE2EDuration="1.521207535s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:45.52091195 +0000 UTC m=+1909.650213899" watchObservedRunningTime="2026-09-18 20:32:45.521207535 +0000 UTC m=+1909.650509484"
Sep 18 20:32:46.338532 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:46.338504    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:46.344620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:46.344595    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:46.507743 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:46.507716    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:46.508547 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:46.508531    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:32:50.064445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.064407    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:32:50.066799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.066783    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.069258 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.069237    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-b-667e4c\""
Sep 18 20:32:50.069373 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.069354    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-b-667e4c-dockercfg-t7q2w\""
Sep 18 20:32:50.083688 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.083660    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:32:50.088130 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.088108    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:32:50.090734 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.090717    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.102096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.102068    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:32:50.126396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.126362    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.126527 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.126490    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.126603 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.126556    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtt26\" (UniqueName: \"kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.126698 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.126684    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vtplj\" (UniqueName: \"kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.228236 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228201    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.228407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228248    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rtt26\" (UniqueName: \"kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.228407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228296    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vtplj\" (UniqueName: \"kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.228407 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228325    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.228966 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228946    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.229037 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.228970    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.236106 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.236083    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vtplj\" (UniqueName: \"kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj\") pod \"payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") " pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.236209 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.236170    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtt26\" (UniqueName: \"kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26\") pod \"payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") " pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.378910 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.378825    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:50.403894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.403867    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:50.654340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.654316    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:32:50.656492 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:32:50.656462    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod192a0556_f879_4fdf_a3ae_31b7410eade1.slice/crio-8abd06895790fa6b8895a0985e5f1040392be5cfc8e7f5053396df7bb3b1d5e1 WatchSource:0}: Error finding container 8abd06895790fa6b8895a0985e5f1040392be5cfc8e7f5053396df7bb3b1d5e1: Status 404 returned error can't find the container with id 8abd06895790fa6b8895a0985e5f1040392be5cfc8e7f5053396df7bb3b1d5e1
Sep 18 20:32:50.689295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:50.689271    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:32:50.691616 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:32:50.691392    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d657faf_03c2_4dd1_a3f5_b121b45592c9.slice/crio-cdf19eb09074d91dbaada097475faaa2c6e98778b74f5ea3b4b93e793a5bf075 WatchSource:0}: Error finding container cdf19eb09074d91dbaada097475faaa2c6e98778b74f5ea3b4b93e793a5bf075: Status 404 returned error can't find the container with id cdf19eb09074d91dbaada097475faaa2c6e98778b74f5ea3b4b93e793a5bf075
Sep 18 20:32:51.539218 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.539169    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" event={"ID":"192a0556-f879-4fdf-a3ae-31b7410eade1","Type":"ContainerStarted","Data":"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"}
Sep 18 20:32:51.539671 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.539242    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" event={"ID":"192a0556-f879-4fdf-a3ae-31b7410eade1","Type":"ContainerStarted","Data":"8abd06895790fa6b8895a0985e5f1040392be5cfc8e7f5053396df7bb3b1d5e1"}
Sep 18 20:32:51.539671 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.539423    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:32:51.541715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.541689    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" event={"ID":"7d657faf-03c2-4dd1-a3f5-b121b45592c9","Type":"ContainerStarted","Data":"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"}
Sep 18 20:32:51.541857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.541735    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" event={"ID":"7d657faf-03c2-4dd1-a3f5-b121b45592c9","Type":"ContainerStarted","Data":"cdf19eb09074d91dbaada097475faaa2c6e98778b74f5ea3b4b93e793a5bf075"}
Sep 18 20:32:51.541857 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.541777    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:32:51.563154 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.563101    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" podStartSLOduration=1.56308205 podStartE2EDuration="1.56308205s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:51.560248412 +0000 UTC m=+1915.689550364" watchObservedRunningTime="2026-09-18 20:32:51.56308205 +0000 UTC m=+1915.692383998"
Sep 18 20:32:51.583447 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:51.583406    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" podStartSLOduration=1.583393499 podStartE2EDuration="1.583393499s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:32:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:32:51.58089532 +0000 UTC m=+1915.710197271" watchObservedRunningTime="2026-09-18 20:32:51.583393499 +0000 UTC m=+1915.712695448"
Sep 18 20:32:58.497705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:32:58.497674    2566 scope.go:117] "RemoveContainer" containerID="10cd24ef46b098836907310200d6fc2e09e8b4aa0ae8789579426c23701e604c"
Sep 18 20:33:02.551229 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:02.551187    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 18 20:33:02.551719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:02.551276    2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 18 20:33:02.552452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:02.552426    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:33:02.552628 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:02.552613    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:33:04.630112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.630050    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c20e633-450a-4045-b617-020dbcebbd3b" containerID="3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be" exitCode=0
Sep 18 20:33:04.630533 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.630119    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerDied","Data":"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"}
Sep 18 20:33:04.630577 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.630543    2566 scope.go:117] "RemoveContainer" containerID="3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"
Sep 18 20:33:04.631677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.631648    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:33:04.631793 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.631733    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:33:04.631793 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:04.631779    2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236204Ki","pods":"250"}
Sep 18 20:33:05.637684 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:05.637649    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerStarted","Data":"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"}
Sep 18 20:33:06.337985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:06.337951    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:33:06.338183 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:06.338153    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 18 20:33:06.338231 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:06.338201    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 18 20:33:07.338433 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:07.338400    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 18 20:33:07.338810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:07.338462    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 18 20:33:08.338316 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:08.338285    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 18 20:33:08.338500 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:08.338339    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 18 20:33:09.338894 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:09.338859    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 18 20:33:09.339339 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:09.338912    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 18 20:33:10.338448 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:10.338418    2566 patch_prober.go:28] interesting pod/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused" start-of-body=
Sep 18 20:33:10.338622 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:10.338482    2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.132.0.97:15021/healthz/ready\": dial tcp 10.132.0.97:15021: connect: connection refused"
Sep 18 20:33:11.341786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:11.341759    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:33:11.342164 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:11.342011    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:33:11.342672 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:11.342653    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:33:17.525783 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:17.525750    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78_7d657faf-03c2-4dd1-a3f5-b121b45592c9/payload-processing/0.log"
Sep 18 20:33:53.760279 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:53.760249    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78_7d657faf-03c2-4dd1-a3f5-b121b45592c9/payload-processing/0.log"
Sep 18 20:33:53.865246 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:53.865215    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78_7d657faf-03c2-4dd1-a3f5-b121b45592c9/payload-processing/0.log"
Sep 18 20:33:58.522260 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:33:58.522225    2566 scope.go:117] "RemoveContainer" containerID="8968d81d9fa9bade9f634bb6323ca3e2c374381237603b88554973750af60670"
Sep 18 20:34:22.921934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:22.921899    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"]
Sep 18 20:34:22.925493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:22.925476    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:22.927748 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:22.927721    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:34:22.927866 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:22.927762    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:34:22.932789 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:22.932763    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"]
Sep 18 20:34:23.047971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:23.047936    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2\" (UID: \"6e5eb844-655d-445e-b8cf-80645d18254e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:23.149001 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:23.148967    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2\" (UID: \"6e5eb844-655d-445e-b8cf-80645d18254e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:23.149701 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:23.149681    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2\" (UID: \"6e5eb844-655d-445e-b8cf-80645d18254e\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:23.237331 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:23.237303    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:23.506747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:23.506722    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"]
Sep 18 20:34:23.508806 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:34:23.508776    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6e5eb844_655d_445e_b8cf_80645d18254e.slice/crio-23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d WatchSource:0}: Error finding container 23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d: Status 404 returned error can't find the container with id 23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d
Sep 18 20:34:24.117276 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:24.117177    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2" event={"ID":"6e5eb844-655d-445e-b8cf-80645d18254e","Type":"ContainerStarted","Data":"27b65a2593b896ea093f5e5deb10e991b310bb37c4b52b7b2169947dd0412eef"}
Sep 18 20:34:24.117276 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:24.117238    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2" event={"ID":"6e5eb844-655d-445e-b8cf-80645d18254e","Type":"ContainerStarted","Data":"23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d"}
Sep 18 20:34:24.132572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:24.132499    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2" podStartSLOduration=2.132482501 podStartE2EDuration="2.132482501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:34:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:34:24.131731382 +0000 UTC m=+2008.261033332" watchObservedRunningTime="2026-09-18 20:34:24.132482501 +0000 UTC m=+2008.261784452"
Sep 18 20:34:25.124191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:25.124149    2566 generic.go:356] "Generic (PLEG): container finished" podID="6e5eb844-655d-445e-b8cf-80645d18254e" containerID="27b65a2593b896ea093f5e5deb10e991b310bb37c4b52b7b2169947dd0412eef" exitCode=0
Sep 18 20:34:25.124654 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:25.124233    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2" event={"ID":"6e5eb844-655d-445e-b8cf-80645d18254e","Type":"ContainerDied","Data":"27b65a2593b896ea093f5e5deb10e991b310bb37c4b52b7b2169947dd0412eef"}
Sep 18 20:34:26.273422 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:26.273388    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:26.382456 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:26.382317    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca\") pod \"6e5eb844-655d-445e-b8cf-80645d18254e\" (UID: \"6e5eb844-655d-445e-b8cf-80645d18254e\") "
Sep 18 20:34:26.382746 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:26.382722    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "6e5eb844-655d-445e-b8cf-80645d18254e" (UID: "6e5eb844-655d-445e-b8cf-80645d18254e"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:34:26.484284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:26.484247    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/6e5eb844-655d-445e-b8cf-80645d18254e-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:34:27.137109 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:27.137048    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2" event={"ID":"6e5eb844-655d-445e-b8cf-80645d18254e","Type":"ContainerDied","Data":"23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d"}
Sep 18 20:34:27.137293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:27.137123    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="23bc8e838c14c671b5c35bea58ae23f113f1aaac04db270d4ea7317814e6ac4d"
Sep 18 20:34:27.137293 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:27.137088    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"
Sep 18 20:34:29.418754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:29.418721    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"]
Sep 18 20:34:29.421107 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:29.421081    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-cleanup-186465-669t2"]
Sep 18 20:34:30.081246 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:30.081214    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6e5eb844-655d-445e-b8cf-80645d18254e" path="/var/lib/kubelet/pods/6e5eb844-655d-445e-b8cf-80645d18254e/volumes"
Sep 18 20:34:39.189151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.189117    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"]
Sep 18 20:34:39.189562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.189548    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6e5eb844-655d-445e-b8cf-80645d18254e" containerName="revoke-keys"
Sep 18 20:34:39.189562 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.189562    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6e5eb844-655d-445e-b8cf-80645d18254e" containerName="revoke-keys"
Sep 18 20:34:39.189676 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.189666    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6e5eb844-655d-445e-b8cf-80645d18254e" containerName="revoke-keys"
Sep 18 20:34:39.192893 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.192876    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:39.195168 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.195146    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:34:39.195288 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.195152    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:34:39.199616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.199592    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"]
Sep 18 20:34:39.314385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.314354    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5\" (UID: \"1e5ec2d2-208f-4d58-85f5-0466ecd9be88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:39.416084 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.416022    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5\" (UID: \"1e5ec2d2-208f-4d58-85f5-0466ecd9be88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:39.416670 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.416651    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5\" (UID: \"1e5ec2d2-208f-4d58-85f5-0466ecd9be88\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:39.505921 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.505894    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:39.784992 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:39.784954    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"]
Sep 18 20:34:39.787705 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:34:39.787675    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e5ec2d2_208f_4d58_85f5_0466ecd9be88.slice/crio-17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb WatchSource:0}: Error finding container 17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb: Status 404 returned error can't find the container with id 17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb
Sep 18 20:34:40.224257 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:40.224215    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5" event={"ID":"1e5ec2d2-208f-4d58-85f5-0466ecd9be88","Type":"ContainerStarted","Data":"7bb91031c6ead526f97227184538ca31181534af8af75e122a88f9d00145dae8"}
Sep 18 20:34:40.224740 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:40.224279    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5" event={"ID":"1e5ec2d2-208f-4d58-85f5-0466ecd9be88","Type":"ContainerStarted","Data":"17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb"}
Sep 18 20:34:40.240954 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:40.240899    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5" podStartSLOduration=1.240882702 podStartE2EDuration="1.240882702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:34:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:34:40.238369614 +0000 UTC m=+2024.367671565" watchObservedRunningTime="2026-09-18 20:34:40.240882702 +0000 UTC m=+2024.370184653"
Sep 18 20:34:41.232268 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:41.232231    2566 generic.go:356] "Generic (PLEG): container finished" podID="1e5ec2d2-208f-4d58-85f5-0466ecd9be88" containerID="7bb91031c6ead526f97227184538ca31181534af8af75e122a88f9d00145dae8" exitCode=0
Sep 18 20:34:41.232656 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:41.232276    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5" event={"ID":"1e5ec2d2-208f-4d58-85f5-0466ecd9be88","Type":"ContainerDied","Data":"7bb91031c6ead526f97227184538ca31181534af8af75e122a88f9d00145dae8"}
Sep 18 20:34:42.378160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:42.378133    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:42.447850 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:42.447814    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca\") pod \"1e5ec2d2-208f-4d58-85f5-0466ecd9be88\" (UID: \"1e5ec2d2-208f-4d58-85f5-0466ecd9be88\") "
Sep 18 20:34:42.448305 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:42.448279    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "1e5ec2d2-208f-4d58-85f5-0466ecd9be88" (UID: "1e5ec2d2-208f-4d58-85f5-0466ecd9be88"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:34:42.548984 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:42.548906    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1e5ec2d2-208f-4d58-85f5-0466ecd9be88-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:34:43.245249 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:43.245218    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"
Sep 18 20:34:43.245249 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:43.245232    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5" event={"ID":"1e5ec2d2-208f-4d58-85f5-0466ecd9be88","Type":"ContainerDied","Data":"17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb"}
Sep 18 20:34:43.245454 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:43.245272    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="17661693361bdf8cc03d4e314ffd4a2037d8cfbec5f12aca0142b51d009fa2cb"
Sep 18 20:34:54.750214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:54.750177    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:34:54.751020 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:54.750965    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" podUID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" containerName="payload-pre-processing" containerID="cri-o://a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976" gracePeriod=30
Sep 18 20:34:54.927736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:54.927702    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"]
Sep 18 20:34:54.931538 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:54.931508    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-a-8cdaea-jl7l5"]
Sep 18 20:34:55.020022 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.019990    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:34:55.179831 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.179798    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume\") pod \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") "
Sep 18 20:34:55.180003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.179853    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-phrhn\" (UniqueName: \"kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn\") pod \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\" (UID: \"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98\") "
Sep 18 20:34:55.180223 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.180202    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" (UID: "2c902c18-15ff-410e-b6a0-dc9ef9d2fd98"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:34:55.181932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.181910    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn" (OuterVolumeSpecName: "kube-api-access-phrhn") pod "2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" (UID: "2c902c18-15ff-410e-b6a0-dc9ef9d2fd98"). InnerVolumeSpecName "kube-api-access-phrhn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:34:55.281496 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.281420    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-phrhn\" (UniqueName: \"kubernetes.io/projected/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-kube-api-access-phrhn\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:34:55.281496 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.281446    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:34:55.313807 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.313773    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" containerID="a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976" exitCode=0
Sep 18 20:34:55.313981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.313827    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"
Sep 18 20:34:55.313981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.313852    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" event={"ID":"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98","Type":"ContainerDied","Data":"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"}
Sep 18 20:34:55.313981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.313908    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4" event={"ID":"2c902c18-15ff-410e-b6a0-dc9ef9d2fd98","Type":"ContainerDied","Data":"c1dd8920b46df632a8323ee620366f861cabdac54823f70e9e3b801b4441165e"}
Sep 18 20:34:55.313981 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.313931    2566 scope.go:117] "RemoveContainer" containerID="a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"
Sep 18 20:34:55.331903 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.331879    2566 scope.go:117] "RemoveContainer" containerID="a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"
Sep 18 20:34:55.332225 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:34:55.332205    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976\": container with ID starting with a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976 not found: ID does not exist" containerID="a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"
Sep 18 20:34:55.332300 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.332232    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976"} err="failed to get container status \"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976\": rpc error: code = NotFound desc = could not find container \"a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976\": container with ID starting with a9058a49e295717c0a51cec002b7e2ec26a67c6bcaf7978465b34b143a8c1976 not found: ID does not exist"
Sep 18 20:34:55.339963 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.339937    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:34:55.343707 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:55.343687    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-a-8cdaea-5548bd6d66-zcts4"]
Sep 18 20:34:56.081413 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:56.081378    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e5ec2d2-208f-4d58-85f5-0466ecd9be88" path="/var/lib/kubelet/pods/1e5ec2d2-208f-4d58-85f5-0466ecd9be88/volumes"
Sep 18 20:34:56.081995 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:34:56.081945    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" path="/var/lib/kubelet/pods/2c902c18-15ff-410e-b6a0-dc9ef9d2fd98/volumes"
Sep 18 20:35:41.324978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.324945    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:35:41.325590 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.325207    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy" containerID="cri-o://f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf" gracePeriod=30
Sep 18 20:35:41.681571 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.681493    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"]
Sep 18 20:35:41.681938 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.681926    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e5ec2d2-208f-4d58-85f5-0466ecd9be88" containerName="revoke-keys"
Sep 18 20:35:41.681985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.681940    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e5ec2d2-208f-4d58-85f5-0466ecd9be88" containerName="revoke-keys"
Sep 18 20:35:41.681985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.681964    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" containerName="payload-pre-processing"
Sep 18 20:35:41.681985 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.681969    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" containerName="payload-pre-processing"
Sep 18 20:35:41.682115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.682038    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="1e5ec2d2-208f-4d58-85f5-0466ecd9be88" containerName="revoke-keys"
Sep 18 20:35:41.682115 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.682050    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c902c18-15ff-410e-b6a0-dc9ef9d2fd98" containerName="payload-pre-processing"
Sep 18 20:35:41.685442 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.685424    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:41.687731 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.687709    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:35:41.687918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.687905    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:35:41.692259 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.692235    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"]
Sep 18 20:35:41.748922 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.748884    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v\" (UID: \"250b3e5f-f141-48fa-8961-a57c6f159d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:41.850143 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.850114    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v\" (UID: \"250b3e5f-f141-48fa-8961-a57c6f159d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:41.850745 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.850724    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v\" (UID: \"250b3e5f-f141-48fa-8961-a57c6f159d3c\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:41.998122 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:41.998086    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:42.270868 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:42.270840    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"]
Sep 18 20:35:42.272706 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:35:42.272682    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod250b3e5f_f141_48fa_8961_a57c6f159d3c.slice/crio-d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f WatchSource:0}: Error finding container d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f: Status 404 returned error can't find the container with id d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f
Sep 18 20:35:42.598805 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:42.598780    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v" event={"ID":"250b3e5f-f141-48fa-8961-a57c6f159d3c","Type":"ContainerStarted","Data":"8ca46e666a162b5ae60c4ca2daea2232e73ae2ebca26204131c2641f36a6985f"}
Sep 18 20:35:42.599210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:42.598826    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v" event={"ID":"250b3e5f-f141-48fa-8961-a57c6f159d3c","Type":"ContainerStarted","Data":"d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f"}
Sep 18 20:35:42.621395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:42.621344    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v" podStartSLOduration=1.621329291 podStartE2EDuration="1.621329291s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:35:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:35:42.618910175 +0000 UTC m=+2086.748212138" watchObservedRunningTime="2026-09-18 20:35:42.621329291 +0000 UTC m=+2086.750631241"
Sep 18 20:35:43.605723 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:43.605687    2566 generic.go:356] "Generic (PLEG): container finished" podID="250b3e5f-f141-48fa-8961-a57c6f159d3c" containerID="8ca46e666a162b5ae60c4ca2daea2232e73ae2ebca26204131c2641f36a6985f" exitCode=0
Sep 18 20:35:43.606112 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:43.605730    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v" event={"ID":"250b3e5f-f141-48fa-8961-a57c6f159d3c","Type":"ContainerDied","Data":"8ca46e666a162b5ae60c4ca2daea2232e73ae2ebca26204131c2641f36a6985f"}
Sep 18 20:35:44.745945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:44.745922    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:44.779513 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:44.779489    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca\") pod \"250b3e5f-f141-48fa-8961-a57c6f159d3c\" (UID: \"250b3e5f-f141-48fa-8961-a57c6f159d3c\") "
Sep 18 20:35:44.779798 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:44.779776    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "250b3e5f-f141-48fa-8961-a57c6f159d3c" (UID: "250b3e5f-f141-48fa-8961-a57c6f159d3c"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:35:44.881070 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:44.880992    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/250b3e5f-f141-48fa-8961-a57c6f159d3c-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:45.619806 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:45.619775    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v" event={"ID":"250b3e5f-f141-48fa-8961-a57c6f159d3c","Type":"ContainerDied","Data":"d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f"}
Sep 18 20:35:45.619986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:45.619810    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"
Sep 18 20:35:45.619986 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:45.619823    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d5c95b79e40f4a1cec3a7321657ee042ce3aa962f3f06c139a9354908619d21f"
Sep 18 20:35:46.589780 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.589730    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:35:46.626824 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.626792    2566 generic.go:356] "Generic (PLEG): container finished" podID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerID="f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf" exitCode=0
Sep 18 20:35:46.626952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.626851    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"
Sep 18 20:35:46.626952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.626872    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerDied","Data":"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"}
Sep 18 20:35:46.626952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.626927    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc" event={"ID":"6209c3ad-0097-430e-84ff-3d22a9542b25","Type":"ContainerDied","Data":"7a125d53edb21425be07adeb03fadc557797a229328781d1728ffdbfce9f09ca"}
Sep 18 20:35:46.626952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.626951    2566 scope.go:117] "RemoveContainer" containerID="f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"
Sep 18 20:35:46.639080 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.639030    2566 scope.go:117] "RemoveContainer" containerID="db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"
Sep 18 20:35:46.647615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.647600    2566 scope.go:117] "RemoveContainer" containerID="f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"
Sep 18 20:35:46.647884 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:35:46.647861    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf\": container with ID starting with f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf not found: ID does not exist" containerID="f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"
Sep 18 20:35:46.647952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.647896    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf"} err="failed to get container status \"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf\": rpc error: code = NotFound desc = could not find container \"f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf\": container with ID starting with f6db9dfbd6eeee6b629f8cbef0a3792b964f469e76ba401e178f3cfb5eab22cf not found: ID does not exist"
Sep 18 20:35:46.647952 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.647921    2566 scope.go:117] "RemoveContainer" containerID="db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"
Sep 18 20:35:46.648202 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:35:46.648178    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789\": container with ID starting with db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789 not found: ID does not exist" containerID="db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"
Sep 18 20:35:46.648296 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.648209    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789"} err="failed to get container status \"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789\": rpc error: code = NotFound desc = could not find container \"db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789\": container with ID starting with db586d1882934b0ab49d27de83ccc4ba59700015c39352e5c6a1e2fadc7dc789 not found: ID does not exist"
Sep 18 20:35:46.700761 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700744    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.700846 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700789    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.700846 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700823    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.700917 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700880    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700915    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.700971    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701006    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dn947\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701017    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:46.701151 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701091    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701155    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo\") pod \"6209c3ad-0097-430e-84ff-3d22a9542b25\" (UID: \"6209c3ad-0097-430e-84ff-3d22a9542b25\") "
Sep 18 20:35:46.701312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701169    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data" (OuterVolumeSpecName: "istio-data") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:46.701312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701282    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:46.701312 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701304    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:35:46.701507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701309    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:46.701636 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701618    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.701704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701642    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.701704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701658    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.701704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701671    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/6209c3ad-0097-430e-84ff-3d22a9542b25-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.701704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.701684    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.703012 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.702988    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token" (OuterVolumeSpecName: "istio-token") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:35:46.703110 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.703050    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:35:46.703572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.703554    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:35:46.703572 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.703561    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947" (OuterVolumeSpecName: "kube-api-access-dn947") pod "6209c3ad-0097-430e-84ff-3d22a9542b25" (UID: "6209c3ad-0097-430e-84ff-3d22a9542b25"). InnerVolumeSpecName "kube-api-access-dn947". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:35:46.802918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.802896    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.802918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.802918    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dn947\" (UniqueName: \"kubernetes.io/projected/6209c3ad-0097-430e-84ff-3d22a9542b25-kube-api-access-dn947\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.803044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.802928    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.803044 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.802937    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/6209c3ad-0097-430e-84ff-3d22a9542b25-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:46.949561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.949532    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:35:46.952684 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:46.952661    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-8cdaea-openshift-default-f95f6c4b6-q4ftc"]
Sep 18 20:35:48.081035 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:48.081007    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" path="/var/lib/kubelet/pods/6209c3ad-0097-430e-84ff-3d22a9542b25/volumes"
Sep 18 20:35:52.108743 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.108709    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:35:52.109238 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.108979    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" podUID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" containerName="payload-processing" containerID="cri-o://dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b" gracePeriod=30
Sep 18 20:35:52.111445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.111421    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:35:52.111683 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.111641    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" podUID="192a0556-f879-4fdf-a3ae-31b7410eade1" containerName="payload-pre-processing" containerID="cri-o://930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179" gracePeriod=30
Sep 18 20:35:52.288295 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.288262    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"]
Sep 18 20:35:52.291843 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.291814    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-ipp-b-667e4c-h4j6v"]
Sep 18 20:35:52.393730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.393705    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:35:52.410906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.410858    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:35:52.454576 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.454542    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume\") pod \"192a0556-f879-4fdf-a3ae-31b7410eade1\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") "
Sep 18 20:35:52.454736 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.454639    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rtt26\" (UniqueName: \"kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26\") pod \"192a0556-f879-4fdf-a3ae-31b7410eade1\" (UID: \"192a0556-f879-4fdf-a3ae-31b7410eade1\") "
Sep 18 20:35:52.454813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.454737    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume\") pod \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") "
Sep 18 20:35:52.454813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.454805    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vtplj\" (UniqueName: \"kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj\") pod \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\" (UID: \"7d657faf-03c2-4dd1-a3f5-b121b45592c9\") "
Sep 18 20:35:52.455301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.454978    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "192a0556-f879-4fdf-a3ae-31b7410eade1" (UID: "192a0556-f879-4fdf-a3ae-31b7410eade1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:35:52.455301 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.455266    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "7d657faf-03c2-4dd1-a3f5-b121b45592c9" (UID: "7d657faf-03c2-4dd1-a3f5-b121b45592c9"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:35:52.455507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.455418    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/7d657faf-03c2-4dd1-a3f5-b121b45592c9-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:52.455507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.455438    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/192a0556-f879-4fdf-a3ae-31b7410eade1-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:52.457160 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.457128    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26" (OuterVolumeSpecName: "kube-api-access-rtt26") pod "192a0556-f879-4fdf-a3ae-31b7410eade1" (UID: "192a0556-f879-4fdf-a3ae-31b7410eade1"). InnerVolumeSpecName "kube-api-access-rtt26". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:35:52.457604 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.457481    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj" (OuterVolumeSpecName: "kube-api-access-vtplj") pod "7d657faf-03c2-4dd1-a3f5-b121b45592c9" (UID: "7d657faf-03c2-4dd1-a3f5-b121b45592c9"). InnerVolumeSpecName "kube-api-access-vtplj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:35:52.556355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.556328    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rtt26\" (UniqueName: \"kubernetes.io/projected/192a0556-f879-4fdf-a3ae-31b7410eade1-kube-api-access-rtt26\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:52.556355 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.556354    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vtplj\" (UniqueName: \"kubernetes.io/projected/7d657faf-03c2-4dd1-a3f5-b121b45592c9-kube-api-access-vtplj\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:35:52.664696 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.664626    2566 generic.go:356] "Generic (PLEG): container finished" podID="192a0556-f879-4fdf-a3ae-31b7410eade1" containerID="930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179" exitCode=0
Sep 18 20:35:52.664696 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.664667    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"
Sep 18 20:35:52.664891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.664712    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" event={"ID":"192a0556-f879-4fdf-a3ae-31b7410eade1","Type":"ContainerDied","Data":"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"}
Sep 18 20:35:52.664891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.664771    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp" event={"ID":"192a0556-f879-4fdf-a3ae-31b7410eade1","Type":"ContainerDied","Data":"8abd06895790fa6b8895a0985e5f1040392be5cfc8e7f5053396df7bb3b1d5e1"}
Sep 18 20:35:52.664891 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.664803    2566 scope.go:117] "RemoveContainer" containerID="930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"
Sep 18 20:35:52.666207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.666184    2566 generic.go:356] "Generic (PLEG): container finished" podID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" containerID="dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b" exitCode=0
Sep 18 20:35:52.666327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.666206    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" event={"ID":"7d657faf-03c2-4dd1-a3f5-b121b45592c9","Type":"ContainerDied","Data":"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"}
Sep 18 20:35:52.666327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.666241    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"
Sep 18 20:35:52.666327 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.666247    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78" event={"ID":"7d657faf-03c2-4dd1-a3f5-b121b45592c9","Type":"ContainerDied","Data":"cdf19eb09074d91dbaada097475faaa2c6e98778b74f5ea3b4b93e793a5bf075"}
Sep 18 20:35:52.675105 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.675042    2566 scope.go:117] "RemoveContainer" containerID="930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"
Sep 18 20:35:52.675358 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:35:52.675341    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179\": container with ID starting with 930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179 not found: ID does not exist" containerID="930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"
Sep 18 20:35:52.675436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.675372    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179"} err="failed to get container status \"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179\": rpc error: code = NotFound desc = could not find container \"930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179\": container with ID starting with 930e5082e078ea3bfbdd47812033c596fb055759c7782b8e29b79f7fd9859179 not found: ID does not exist"
Sep 18 20:35:52.675436 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.675395    2566 scope.go:117] "RemoveContainer" containerID="dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"
Sep 18 20:35:52.683808 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.683794    2566 scope.go:117] "RemoveContainer" containerID="dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"
Sep 18 20:35:52.684082 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:35:52.684051    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b\": container with ID starting with dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b not found: ID does not exist" containerID="dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"
Sep 18 20:35:52.684155 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.684090    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b"} err="failed to get container status \"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b\": rpc error: code = NotFound desc = could not find container \"dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b\": container with ID starting with dc9c74532ddc956bb8285300f34ce4a07e8586ce51f62432495d24c3c15d414b not found: ID does not exist"
Sep 18 20:35:52.689544 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.689523    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:35:52.691568 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.691549    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ipp-b-667e4c-5c78db7f98-zflkp"]
Sep 18 20:35:52.701899 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.701879    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:35:52.708046 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:52.708028    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-b-667e4c-7dd5756c65-qvf78"]
Sep 18 20:35:54.081213 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:54.081107    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="192a0556-f879-4fdf-a3ae-31b7410eade1" path="/var/lib/kubelet/pods/192a0556-f879-4fdf-a3ae-31b7410eade1/volumes"
Sep 18 20:35:54.081827 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:54.081796    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="250b3e5f-f141-48fa-8961-a57c6f159d3c" path="/var/lib/kubelet/pods/250b3e5f-f141-48fa-8961-a57c6f159d3c/volumes"
Sep 18 20:35:54.082586 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:54.082554    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" path="/var/lib/kubelet/pods/7d657faf-03c2-4dd1-a3f5-b121b45592c9/volumes"
Sep 18 20:35:58.232715 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.232685    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:35:58.238584 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.238562    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:35:58.562783 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.562713    2566 scope.go:117] "RemoveContainer" containerID="ff6793181a642d73501717c897a7b907822d2287e395f0ff7d0f40a1a59daa91"
Sep 18 20:35:58.572087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.572047    2566 scope.go:117] "RemoveContainer" containerID="973ca71a49b06ed1158747a6b5089c2f7ee6fae3ebcbe9c850124291f0b7bbe6"
Sep 18 20:35:58.861234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.861157    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:35:58.861412 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:58.861388    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy" containerID="cri-o://e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6" gracePeriod=30
Sep 18 20:35:59.201995 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.201915    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"]
Sep 18 20:35:59.202408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202384    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.202408 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202406    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202426    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" containerName="payload-processing"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202435    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" containerName="payload-processing"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202459    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202468    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202481    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="250b3e5f-f141-48fa-8961-a57c6f159d3c" containerName="revoke-keys"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202490    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="250b3e5f-f141-48fa-8961-a57c6f159d3c" containerName="revoke-keys"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202504    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="192a0556-f879-4fdf-a3ae-31b7410eade1" containerName="payload-pre-processing"
Sep 18 20:35:59.202575 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202512    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="192a0556-f879-4fdf-a3ae-31b7410eade1" containerName="payload-pre-processing"
Sep 18 20:35:59.202812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202617    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="250b3e5f-f141-48fa-8961-a57c6f159d3c" containerName="revoke-keys"
Sep 18 20:35:59.202812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202628    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="192a0556-f879-4fdf-a3ae-31b7410eade1" containerName="payload-pre-processing"
Sep 18 20:35:59.202812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202634    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d657faf-03c2-4dd1-a3f5-b121b45592c9" containerName="payload-processing"
Sep 18 20:35:59.202812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202641    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.202812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.202650    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="6209c3ad-0097-430e-84ff-3d22a9542b25" containerName="istio-proxy"
Sep 18 20:35:59.207530 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.207513    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:35:59.210299 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.210269    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:35:59.210421 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.210333    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:35:59.212452 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.212433    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"]
Sep 18 20:35:59.311395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.311366    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l\" (UID: \"3813b0da-04e2-4e56-a154-fdcb6cffedbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:35:59.412754 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.412724    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l\" (UID: \"3813b0da-04e2-4e56-a154-fdcb6cffedbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:35:59.413362 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.413342    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l\" (UID: \"3813b0da-04e2-4e56-a154-fdcb6cffedbe\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:35:59.521204 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.521179    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:35:59.790530 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:35:59.790505    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"]
Sep 18 20:35:59.792841 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:35:59.792808    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3813b0da_04e2_4e56_a154_fdcb6cffedbe.slice/crio-1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6 WatchSource:0}: Error finding container 1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6: Status 404 returned error can't find the container with id 1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6
Sep 18 20:36:00.716384 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:00.716314    2566 generic.go:356] "Generic (PLEG): container finished" podID="3813b0da-04e2-4e56-a154-fdcb6cffedbe" containerID="84672156bc58d0ebdb64e2dea9b36e8f0fe5b89562f317cbcf04b3e046c10a52" exitCode=0
Sep 18 20:36:00.716689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:00.716384    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l" event={"ID":"3813b0da-04e2-4e56-a154-fdcb6cffedbe","Type":"ContainerDied","Data":"84672156bc58d0ebdb64e2dea9b36e8f0fe5b89562f317cbcf04b3e046c10a52"}
Sep 18 20:36:00.716689 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:00.716422    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l" event={"ID":"3813b0da-04e2-4e56-a154-fdcb6cffedbe","Type":"ContainerStarted","Data":"1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6"}
Sep 18 20:36:01.855697 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:01.855670    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:36:01.935732 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:01.935704    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca\") pod \"3813b0da-04e2-4e56-a154-fdcb6cffedbe\" (UID: \"3813b0da-04e2-4e56-a154-fdcb6cffedbe\") "
Sep 18 20:36:01.936042 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:01.936021    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "3813b0da-04e2-4e56-a154-fdcb6cffedbe" (UID: "3813b0da-04e2-4e56-a154-fdcb6cffedbe"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:02.037193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:02.037114    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/3813b0da-04e2-4e56-a154-fdcb6cffedbe-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:02.734971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:02.734939    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l" event={"ID":"3813b0da-04e2-4e56-a154-fdcb6cffedbe","Type":"ContainerDied","Data":"1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6"}
Sep 18 20:36:02.734971 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:02.734961    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"
Sep 18 20:36:02.735201 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:02.735013    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1e0be5c0d3c432b8160b5e75d07bfc45e3e239e0519efa3f5c6e8cc3f58433f6"
Sep 18 20:36:04.115328 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.115301    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:36:04.254772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254737    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8txjl\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.254932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254785    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.254932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254817    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.254932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254838    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.254932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254887    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.254932 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254902    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.255226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254948    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.255226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254964    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.255226 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.254991    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token\") pod \"2c20e633-450a-4045-b617-020dbcebbd3b\" (UID: \"2c20e633-450a-4045-b617-020dbcebbd3b\") "
Sep 18 20:36:04.255382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255227    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:04.255382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255246    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:04.255382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255321    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:04.255382 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255336    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.255584 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255399    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:04.255584 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.255489    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data" (OuterVolumeSpecName: "istio-data") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:04.257004 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.256974    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl" (OuterVolumeSpecName: "kube-api-access-8txjl") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "kube-api-access-8txjl". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:04.257314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.257291    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:36:04.257401 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.257354    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:04.257478 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.257463    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token" (OuterVolumeSpecName: "istio-token") pod "2c20e633-450a-4045-b617-020dbcebbd3b" (UID: "2c20e633-450a-4045-b617-020dbcebbd3b"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:04.356677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356635    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/2c20e633-450a-4045-b617-020dbcebbd3b-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356672    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356686    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356699    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356710    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/2c20e633-450a-4045-b617-020dbcebbd3b-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356721    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356733    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8txjl\" (UniqueName: \"kubernetes.io/projected/2c20e633-450a-4045-b617-020dbcebbd3b-kube-api-access-8txjl\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.356904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.356745    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/2c20e633-450a-4045-b617-020dbcebbd3b-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:04.748359 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.748322    2566 generic.go:356] "Generic (PLEG): container finished" podID="2c20e633-450a-4045-b617-020dbcebbd3b" containerID="e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6" exitCode=0
Sep 18 20:36:04.748495 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.748377    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"
Sep 18 20:36:04.748495 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.748415    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerDied","Data":"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"}
Sep 18 20:36:04.748495 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.748480    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh" event={"ID":"2c20e633-450a-4045-b617-020dbcebbd3b","Type":"ContainerDied","Data":"90a0fb7c564f8070c54ea6f90a3a6aa4decb7a6f2796290e6896bc36cf3e482a"}
Sep 18 20:36:04.748615 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.748504    2566 scope.go:117] "RemoveContainer" containerID="e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"
Sep 18 20:36:04.758519 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.758504    2566 scope.go:117] "RemoveContainer" containerID="3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"
Sep 18 20:36:04.767012 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.766994    2566 scope.go:117] "RemoveContainer" containerID="e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"
Sep 18 20:36:04.767303 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:04.767285    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6\": container with ID starting with e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6 not found: ID does not exist" containerID="e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"
Sep 18 20:36:04.767350 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.767310    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6"} err="failed to get container status \"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6\": rpc error: code = NotFound desc = could not find container \"e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6\": container with ID starting with e4dbb33f238d291ea724a31730a4b89843594eff29710ef852af1e6359c145a6 not found: ID does not exist"
Sep 18 20:36:04.767350 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.767325    2566 scope.go:117] "RemoveContainer" containerID="3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"
Sep 18 20:36:04.767550 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:04.767534    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be\": container with ID starting with 3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be not found: ID does not exist" containerID="3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"
Sep 18 20:36:04.767597 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.767554    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be"} err="failed to get container status \"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be\": rpc error: code = NotFound desc = could not find container \"3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be\": container with ID starting with 3366b4424ab0b6e5070382024dfd6365745a1c834c3656b90337079c25be79be not found: ID does not exist"
Sep 18 20:36:04.771313 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.771292    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:36:04.774224 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:04.774207    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-667e4c-openshift-default-cbc5ff7d6-jz7sh"]
Sep 18 20:36:06.080734 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:06.080701    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" path="/var/lib/kubelet/pods/2c20e633-450a-4045-b617-020dbcebbd3b/volumes"
Sep 18 20:36:09.729964 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.729928    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"]
Sep 18 20:36:09.731937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.731911    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-bab50d-29829390-h2kkk"]
Sep 18 20:36:09.755723 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.755697    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:36:09.756003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.755976    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" podUID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" containerName="payload-processing" containerID="cri-o://c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44" gracePeriod=30
Sep 18 20:36:09.937694 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.937661    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"]
Sep 18 20:36:09.940033 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:09.939997    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-a-w2-bab50d-wqf2l"]
Sep 18 20:36:10.020191 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.020168    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:36:10.085214 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.085170    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3813b0da-04e2-4e56-a154-fdcb6cffedbe" path="/var/lib/kubelet/pods/3813b0da-04e2-4e56-a154-fdcb6cffedbe/volumes"
Sep 18 20:36:10.085751 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.085726    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6" path="/var/lib/kubelet/pods/cc6c0d1b-bdef-41a6-87ae-f0fbf07ea1c6/volumes"
Sep 18 20:36:10.105309 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.105287    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-k46g7\" (UniqueName: \"kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7\") pod \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") "
Sep 18 20:36:10.105447 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.105369    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume\") pod \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\" (UID: \"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd\") "
Sep 18 20:36:10.105791 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.105762    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" (UID: "ea4c0892-03bf-4fd8-95fb-cf019eb8abfd"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:10.107351 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.107325    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7" (OuterVolumeSpecName: "kube-api-access-k46g7") pod "ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" (UID: "ea4c0892-03bf-4fd8-95fb-cf019eb8abfd"). InnerVolumeSpecName "kube-api-access-k46g7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:10.206705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.206675    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-k46g7\" (UniqueName: \"kubernetes.io/projected/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-kube-api-access-k46g7\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.206705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.206701    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:10.790836 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.790803    2566 generic.go:356] "Generic (PLEG): container finished" podID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" containerID="c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44" exitCode=0
Sep 18 20:36:10.791264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.790847    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"
Sep 18 20:36:10.791264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.790860    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" event={"ID":"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd","Type":"ContainerDied","Data":"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"}
Sep 18 20:36:10.791264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.790901    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r" event={"ID":"ea4c0892-03bf-4fd8-95fb-cf019eb8abfd","Type":"ContainerDied","Data":"f2cd2e70b6ba9193cdd5e60bfc986acb9c65f16b2696eb6ca36a211ecc30eaa6"}
Sep 18 20:36:10.791264 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.790923    2566 scope.go:117] "RemoveContainer" containerID="c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"
Sep 18 20:36:10.800813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.800795    2566 scope.go:117] "RemoveContainer" containerID="c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"
Sep 18 20:36:10.801099 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:10.801081    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44\": container with ID starting with c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44 not found: ID does not exist" containerID="c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"
Sep 18 20:36:10.801162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.801108    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44"} err="failed to get container status \"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44\": rpc error: code = NotFound desc = could not find container \"c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44\": container with ID starting with c7023a0516b03721ddf1a753a140a55e173378a954ff9f6fd017f30d54f6bc44 not found: ID does not exist"
Sep 18 20:36:10.812451 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.812428    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:36:10.816904 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:10.816883    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-bab50d-7cc7b9fbcc-ldg6r"]
Sep 18 20:36:12.081179 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:12.081151    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" path="/var/lib/kubelet/pods/ea4c0892-03bf-4fd8-95fb-cf019eb8abfd/volumes"
Sep 18 20:36:16.582712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.582667    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:36:16.583149 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.582937    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy" containerID="cri-o://bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375" gracePeriod=30
Sep 18 20:36:16.932532 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932442    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"]
Sep 18 20:36:16.932863 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932850    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.932911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932864    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.932911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932882    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" containerName="payload-processing"
Sep 18 20:36:16.932911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932888    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" containerName="payload-processing"
Sep 18 20:36:16.932911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932895    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.932911 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932900    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932922    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3813b0da-04e2-4e56-a154-fdcb6cffedbe" containerName="revoke-keys"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932929    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="3813b0da-04e2-4e56-a154-fdcb6cffedbe" containerName="revoke-keys"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932990    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="3813b0da-04e2-4e56-a154-fdcb6cffedbe" containerName="revoke-keys"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.932998    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.933005    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2c20e633-450a-4045-b617-020dbcebbd3b" containerName="istio-proxy"
Sep 18 20:36:16.933096 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.933013    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="ea4c0892-03bf-4fd8-95fb-cf019eb8abfd" containerName="payload-processing"
Sep 18 20:36:16.937668 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.937648    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:16.939957 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.939931    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-5nwx7\""
Sep 18 20:36:16.940092 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.939977    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 18 20:36:16.943960 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:16.943923    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"]
Sep 18 20:36:17.071759 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.071723    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg\" (UID: \"680237ed-08af-4e1e-9832-80bc30107621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:17.172660 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.172622    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg\" (UID: \"680237ed-08af-4e1e-9832-80bc30107621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:17.173331 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.173310    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg\" (UID: \"680237ed-08af-4e1e-9832-80bc30107621\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:17.249969 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.249935    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:17.526030 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.525996    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"]
Sep 18 20:36:17.527985 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:36:17.527957    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod680237ed_08af_4e1e_9832_80bc30107621.slice/crio-d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88 WatchSource:0}: Error finding container d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88: Status 404 returned error can't find the container with id d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88
Sep 18 20:36:17.832278 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:17.832202    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg" event={"ID":"680237ed-08af-4e1e-9832-80bc30107621","Type":"ContainerStarted","Data":"d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88"}
Sep 18 20:36:18.840139 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:18.840100    2566 generic.go:356] "Generic (PLEG): container finished" podID="680237ed-08af-4e1e-9832-80bc30107621" containerID="49b84d6d7e3e786d194a5d0c48a3f3418c06a41b486a93ebf8b053aa0c1d8fe0" exitCode=0
Sep 18 20:36:18.840515 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:18.840217    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg" event={"ID":"680237ed-08af-4e1e-9832-80bc30107621","Type":"ContainerDied","Data":"49b84d6d7e3e786d194a5d0c48a3f3418c06a41b486a93ebf8b053aa0c1d8fe0"}
Sep 18 20:36:19.975739 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:19.975717    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:20.101598 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.101530    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca\") pod \"680237ed-08af-4e1e-9832-80bc30107621\" (UID: \"680237ed-08af-4e1e-9832-80bc30107621\") "
Sep 18 20:36:20.101864 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.101843    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "680237ed-08af-4e1e-9832-80bc30107621" (UID: "680237ed-08af-4e1e-9832-80bc30107621"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:20.202769 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.202737    2566 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/680237ed-08af-4e1e-9832-80bc30107621-maas-api-service-ca\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:20.853352 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.853316    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg" event={"ID":"680237ed-08af-4e1e-9832-80bc30107621","Type":"ContainerDied","Data":"d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88"}
Sep 18 20:36:20.853561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.853380    2566 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d186ced0456d5201990fa789fecf1cb6ad1d3afdd08fb9c3b6583766d7bf5b88"
Sep 18 20:36:20.853561 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:20.853323    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"
Sep 18 20:36:21.833281 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.833260    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:36:21.861897 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.861837    2566 generic.go:356] "Generic (PLEG): container finished" podID="e584c927-ba85-4533-ab33-2b496e361960" containerID="bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375" exitCode=0
Sep 18 20:36:21.862007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.861897    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"
Sep 18 20:36:21.862007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.861909    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerDied","Data":"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"}
Sep 18 20:36:21.862007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.861966    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl" event={"ID":"e584c927-ba85-4533-ab33-2b496e361960","Type":"ContainerDied","Data":"73fec062403c7907f185937c2509316d04d86ce4d2651c809f55dd95252a3d22"}
Sep 18 20:36:21.862007 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.861991    2566 scope.go:117] "RemoveContainer" containerID="bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"
Sep 18 20:36:21.874104 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.874085    2566 scope.go:117] "RemoveContainer" containerID="6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"
Sep 18 20:36:21.882370 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.882353    2566 scope.go:117] "RemoveContainer" containerID="bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"
Sep 18 20:36:21.882658 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:21.882638    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375\": container with ID starting with bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375 not found: ID does not exist" containerID="bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"
Sep 18 20:36:21.882753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.882665    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375"} err="failed to get container status \"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375\": rpc error: code = NotFound desc = could not find container \"bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375\": container with ID starting with bf098bb970dd3864c1792eaa97eb5c1f538794e8ffc6f039f2d8384f8fe62375 not found: ID does not exist"
Sep 18 20:36:21.882753 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.882683    2566 scope.go:117] "RemoveContainer" containerID="6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"
Sep 18 20:36:21.882956 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:21.882937    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc\": container with ID starting with 6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc not found: ID does not exist" containerID="6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"
Sep 18 20:36:21.883025 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:21.882968    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc"} err="failed to get container status \"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc\": rpc error: code = NotFound desc = could not find container \"6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc\": container with ID starting with 6bc787f6533efbe591fed1ed9a5624d5a1ef19d8918fe942cfd3aa9f1b495abc not found: ID does not exist"
Sep 18 20:36:22.020125 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020103    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020235 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020139    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fdjw\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020235 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020158    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020235 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020180    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020235 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020227    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020254    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020283    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020338    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020371    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo\") pod \"e584c927-ba85-4533-ab33-2b496e361960\" (UID: \"e584c927-ba85-4533-ab33-2b496e361960\") "
Sep 18 20:36:22.020488 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020403    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:22.020795 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020559    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data" (OuterVolumeSpecName: "istio-data") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:22.020795 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020732    2566 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-credential-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.020795 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020751    2566 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-data\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.020795 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020753    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:22.021009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020839    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:22.021009 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.020933    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:22.022679 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.022651    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token" (OuterVolumeSpecName: "istio-token") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:22.022772 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.022709    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw" (OuterVolumeSpecName: "kube-api-access-5fdjw") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "kube-api-access-5fdjw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:22.022854 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.022834    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:36:22.022951 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.022934    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "e584c927-ba85-4533-ab33-2b496e361960" (UID: "e584c927-ba85-4533-ab33-2b496e361960"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121727    2566 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-istio-envoy\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121747    2566 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121755    2566 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/e584c927-ba85-4533-ab33-2b496e361960-istiod-ca-cert\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121763    2566 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/e584c927-ba85-4533-ab33-2b496e361960-istio-podinfo\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121770    2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-istio-token\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.121775 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121778    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fdjw\" (UniqueName: \"kubernetes.io/projected/e584c927-ba85-4533-ab33-2b496e361960-kube-api-access-5fdjw\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.122010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.121786    2566 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/e584c927-ba85-4533-ab33-2b496e361960-workload-socket\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:22.179249 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.179225    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:36:22.182589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:22.182570    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-bab50d-openshift-default-555dd5c6cd-24xpl"]
Sep 18 20:36:24.081180 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:24.081149    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e584c927-ba85-4533-ab33-2b496e361960" path="/var/lib/kubelet/pods/e584c927-ba85-4533-ab33-2b496e361960/volumes"
Sep 18 20:36:27.520499 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.520463    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:36:27.520945 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.520747    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" podUID="094df03c-f1a1-4a3b-9032-3eeb138f1022" containerName="payload-pre-processing" containerID="cri-o://70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393" gracePeriod=30
Sep 18 20:36:27.687010 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.686930    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"]
Sep 18 20:36:27.689110 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.689082    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-shared-b-w2-35558b-wmxqg"]
Sep 18 20:36:27.779799 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.779737    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:36:27.877626 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.877593    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tthnr\" (UniqueName: \"kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr\") pod \"094df03c-f1a1-4a3b-9032-3eeb138f1022\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") "
Sep 18 20:36:27.877813 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.877698    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume\") pod \"094df03c-f1a1-4a3b-9032-3eeb138f1022\" (UID: \"094df03c-f1a1-4a3b-9032-3eeb138f1022\") "
Sep 18 20:36:27.878107 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.878082    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "094df03c-f1a1-4a3b-9032-3eeb138f1022" (UID: "094df03c-f1a1-4a3b-9032-3eeb138f1022"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 18 20:36:27.879706 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.879687    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr" (OuterVolumeSpecName: "kube-api-access-tthnr") pod "094df03c-f1a1-4a3b-9032-3eeb138f1022" (UID: "094df03c-f1a1-4a3b-9032-3eeb138f1022"). InnerVolumeSpecName "kube-api-access-tthnr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:36:27.897496 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.897464    2566 generic.go:356] "Generic (PLEG): container finished" podID="094df03c-f1a1-4a3b-9032-3eeb138f1022" containerID="70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393" exitCode=0
Sep 18 20:36:27.897602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.897506    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"
Sep 18 20:36:27.897602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.897544    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" event={"ID":"094df03c-f1a1-4a3b-9032-3eeb138f1022","Type":"ContainerDied","Data":"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"}
Sep 18 20:36:27.897602 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.897596    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2" event={"ID":"094df03c-f1a1-4a3b-9032-3eeb138f1022","Type":"ContainerDied","Data":"95ebe8df54bbfc13c9b2f303ca0a12aabfcc25c4b7760174d9d8f3e1c2af4f7e"}
Sep 18 20:36:27.897737 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.897618    2566 scope.go:117] "RemoveContainer" containerID="70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"
Sep 18 20:36:27.906542 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.906524    2566 scope.go:117] "RemoveContainer" containerID="70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"
Sep 18 20:36:27.906809 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:36:27.906791    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393\": container with ID starting with 70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393 not found: ID does not exist" containerID="70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"
Sep 18 20:36:27.906871 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.906820    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393"} err="failed to get container status \"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393\": rpc error: code = NotFound desc = could not find container \"70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393\": container with ID starting with 70649c553e18253bad5f0d8a6de0b250b111591f73604dfcc8fb215aab897393 not found: ID does not exist"
Sep 18 20:36:27.917716 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.917694    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:36:27.921343 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.921320    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-b-w2-35558b-56b894444b-9n9q2"]
Sep 18 20:36:27.978664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.978636    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tthnr\" (UniqueName: \"kubernetes.io/projected/094df03c-f1a1-4a3b-9032-3eeb138f1022-kube-api-access-tthnr\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:27.978664 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:27.978664    2566 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/094df03c-f1a1-4a3b-9032-3eeb138f1022-plugins-config-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:36:28.086003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:28.085925    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="094df03c-f1a1-4a3b-9032-3eeb138f1022" path="/var/lib/kubelet/pods/094df03c-f1a1-4a3b-9032-3eeb138f1022/volumes"
Sep 18 20:36:28.086332 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:28.086316    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="680237ed-08af-4e1e-9832-80bc30107621" path="/var/lib/kubelet/pods/680237ed-08af-4e1e-9832-80bc30107621/volumes"
Sep 18 20:36:58.609796 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:58.609760    2566 scope.go:117] "RemoveContainer" containerID="cd7da44b8ea5888ffd1498e95d7cc9ad44e363bda19b11b70d15a70621937d8d"
Sep 18 20:36:58.618967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:58.618945    2566 scope.go:117] "RemoveContainer" containerID="a7174dbef0b737f582b4d65af104421418824d637f2513a505cea6c7546f8bac"
Sep 18 20:36:58.627395 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:58.627370    2566 scope.go:117] "RemoveContainer" containerID="00a4b923895e2cc61b72f13968b40f3434a6d6b38cce3e44807f4227fbb21608"
Sep 18 20:36:58.635203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:36:58.635184    2566 scope.go:117] "RemoveContainer" containerID="9fc9925677f44ce338cd79d7857c3d2aa465d4103a61ae725edf36d8410ded49"
Sep 18 20:39:18.903511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:18.903476    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:39:18.904014 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:18.903685    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" podUID="918a35d0-3c67-4f38-b13a-213a797b6aa2" containerName="manager" containerID="cri-o://1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae" gracePeriod=10
Sep 18 20:39:19.157761 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.157693    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:39:19.193252 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.193225    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume\") pod \"918a35d0-3c67-4f38-b13a-213a797b6aa2\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") "
Sep 18 20:39:19.193410 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.193268    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-frsjs\" (UniqueName: \"kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs\") pod \"918a35d0-3c67-4f38-b13a-213a797b6aa2\" (UID: \"918a35d0-3c67-4f38-b13a-213a797b6aa2\") "
Sep 18 20:39:19.193673 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.193642    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "918a35d0-3c67-4f38-b13a-213a797b6aa2" (UID: "918a35d0-3c67-4f38-b13a-213a797b6aa2"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:39:19.195374 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.195350    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs" (OuterVolumeSpecName: "kube-api-access-frsjs") pod "918a35d0-3c67-4f38-b13a-213a797b6aa2" (UID: "918a35d0-3c67-4f38-b13a-213a797b6aa2"). InnerVolumeSpecName "kube-api-access-frsjs". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:39:19.294130 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.294103    2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/918a35d0-3c67-4f38-b13a-213a797b6aa2-extensions-socket-volume\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:39:19.294130 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.294126    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-frsjs\" (UniqueName: \"kubernetes.io/projected/918a35d0-3c67-4f38-b13a-213a797b6aa2-kube-api-access-frsjs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:39:19.830426 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.830390    2566 generic.go:356] "Generic (PLEG): container finished" podID="918a35d0-3c67-4f38-b13a-213a797b6aa2" containerID="1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae" exitCode=0
Sep 18 20:39:19.830620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.830436    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"
Sep 18 20:39:19.830620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.830485    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" event={"ID":"918a35d0-3c67-4f38-b13a-213a797b6aa2","Type":"ContainerDied","Data":"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"}
Sep 18 20:39:19.830620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.830555    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9" event={"ID":"918a35d0-3c67-4f38-b13a-213a797b6aa2","Type":"ContainerDied","Data":"b2df7770b13d034915ae9aa261e9a15eb2c1a5d60f0f5f7561d12f658e201117"}
Sep 18 20:39:19.830620 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.830590    2566 scope.go:117] "RemoveContainer" containerID="1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"
Sep 18 20:39:19.841026 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.841006    2566 scope.go:117] "RemoveContainer" containerID="1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"
Sep 18 20:39:19.841344 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:39:19.841325    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae\": container with ID starting with 1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae not found: ID does not exist" containerID="1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"
Sep 18 20:39:19.841399 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.841353    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae"} err="failed to get container status \"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae\": rpc error: code = NotFound desc = could not find container \"1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae\": container with ID starting with 1bfc14499679ed58967193d8f9884c3fd24e6a5ec1d058cdc3a8957ebfef72ae not found: ID does not exist"
Sep 18 20:39:19.852883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.852860    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:39:19.856656 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:19.856635    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-999v9"]
Sep 18 20:39:20.079934 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:39:20.079906    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="918a35d0-3c67-4f38-b13a-213a797b6aa2" path="/var/lib/kubelet/pods/918a35d0-3c67-4f38-b13a-213a797b6aa2/volumes"
Sep 18 20:40:19.599876 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.599843    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"]
Sep 18 20:40:19.600284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600250    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="680237ed-08af-4e1e-9832-80bc30107621" containerName="revoke-keys"
Sep 18 20:40:19.600284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600262    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="680237ed-08af-4e1e-9832-80bc30107621" containerName="revoke-keys"
Sep 18 20:40:19.600284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600271    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="918a35d0-3c67-4f38-b13a-213a797b6aa2" containerName="manager"
Sep 18 20:40:19.600284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600276    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="918a35d0-3c67-4f38-b13a-213a797b6aa2" containerName="manager"
Sep 18 20:40:19.600284 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600284    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600290    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600300    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="094df03c-f1a1-4a3b-9032-3eeb138f1022" containerName="payload-pre-processing"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600306    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="094df03c-f1a1-4a3b-9032-3eeb138f1022" containerName="payload-pre-processing"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600373    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600381    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="094df03c-f1a1-4a3b-9032-3eeb138f1022" containerName="payload-pre-processing"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600387    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="680237ed-08af-4e1e-9832-80bc30107621" containerName="revoke-keys"
Sep 18 20:40:19.600445 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.600394    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="918a35d0-3c67-4f38-b13a-213a797b6aa2" containerName="manager"
Sep 18 20:40:19.604593 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.604577    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.606747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.606729    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tkd9r\""
Sep 18 20:40:19.614539 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.614520    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"]
Sep 18 20:40:19.685708 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.685680    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gml6q\" (UniqueName: \"kubernetes.io/projected/69416089-53df-4ad0-8fea-bce2b992a351-kube-api-access-gml6q\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.685800 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.685715    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69416089-53df-4ad0-8fea-bce2b992a351-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.787144 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.787112    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gml6q\" (UniqueName: \"kubernetes.io/projected/69416089-53df-4ad0-8fea-bce2b992a351-kube-api-access-gml6q\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.787256 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.787156    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69416089-53df-4ad0-8fea-bce2b992a351-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.787533 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.787512    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/69416089-53df-4ad0-8fea-bce2b992a351-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.795961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.795940    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gml6q\" (UniqueName: \"kubernetes.io/projected/69416089-53df-4ad0-8fea-bce2b992a351-kube-api-access-gml6q\") pod \"kuadrant-operator-controller-manager-5769f99d9f-hcj4p\" (UID: \"69416089-53df-4ad0-8fea-bce2b992a351\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:19.915343 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:19.915277    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:20.195635 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:20.195564    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"]
Sep 18 20:40:20.199505 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:40:20.199475    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69416089_53df_4ad0_8fea_bce2b992a351.slice/crio-4947704c6bb65172e278a2390b5e1302a6e6ddfde2dbcacbf697b72f6d1be224 WatchSource:0}: Error finding container 4947704c6bb65172e278a2390b5e1302a6e6ddfde2dbcacbf697b72f6d1be224: Status 404 returned error can't find the container with id 4947704c6bb65172e278a2390b5e1302a6e6ddfde2dbcacbf697b72f6d1be224
Sep 18 20:40:21.155474 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:21.155448    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p" event={"ID":"69416089-53df-4ad0-8fea-bce2b992a351","Type":"ContainerStarted","Data":"5b060071e440c59b89e5367453f7b1f0bc6345cc161d062e07e8750f9132f0ce"}
Sep 18 20:40:21.155906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:21.155493    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p" event={"ID":"69416089-53df-4ad0-8fea-bce2b992a351","Type":"ContainerStarted","Data":"4947704c6bb65172e278a2390b5e1302a6e6ddfde2dbcacbf697b72f6d1be224"}
Sep 18 20:40:21.155906 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:21.155543    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:21.179414 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:21.179371    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p" podStartSLOduration=2.179358933 podStartE2EDuration="2.179358933s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:40:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:40:21.176654707 +0000 UTC m=+2365.305956656" watchObservedRunningTime="2026-09-18 20:40:21.179358933 +0000 UTC m=+2365.308660882"
Sep 18 20:40:32.163517 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:32.163485    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5769f99d9f-hcj4p"
Sep 18 20:40:58.266939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:58.266909    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:40:58.275432 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:58.275411    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:40:58.714008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:58.713979    2566 scope.go:117] "RemoveContainer" containerID="27b65a2593b896ea093f5e5deb10e991b310bb37c4b52b7b2169947dd0412eef"
Sep 18 20:40:58.721798 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:40:58.721779    2566 scope.go:117] "RemoveContainer" containerID="7bb91031c6ead526f97227184538ca31181534af8af75e122a88f9d00145dae8"
Sep 18 20:41:13.481069 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.481029    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:41:13.481684 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.481662    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:41:13.481728 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.481688    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:41:13.481820 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.481807    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="e584c927-ba85-4533-ab33-2b496e361960" containerName="istio-proxy"
Sep 18 20:41:13.486704 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.486680    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.489091 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.489042    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-a-7e57cbca-kserve-self-signed-certs\""
Sep 18 20:41:13.493812 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.493789    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:41:13.637928 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.637899    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.638087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.637937    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.638087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.637974    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-49j6t\" (UniqueName: \"kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.638087 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.638036    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.638212 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.638133    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.638212 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.638155    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739456    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739493    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739541 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739524    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739601    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739656    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-49j6t\" (UniqueName: \"kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739692    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.739951 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739875    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.740008 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.739954    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.740255 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.740229    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.742108 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.742087    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.742677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.742654    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.749923 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.749897    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-49j6t\" (UniqueName: \"kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t\") pod \"e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") " pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:13.798560 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:13.798535    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:14.079455 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:41:14.079426    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb61a78fe_e7a3_4df3_9de0_2221c42f54d0.slice/crio-1535b3a58e038238933d474ac19e6fba26fb532993dbdbee4b6ea120e187c64b WatchSource:0}: Error finding container 1535b3a58e038238933d474ac19e6fba26fb532993dbdbee4b6ea120e187c64b: Status 404 returned error can't find the container with id 1535b3a58e038238933d474ac19e6fba26fb532993dbdbee4b6ea120e187c64b
Sep 18 20:41:14.081234 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:14.081212    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:41:14.439634 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:14.439539    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" event={"ID":"b61a78fe-e7a3-4df3-9de0-2221c42f54d0","Type":"ContainerStarted","Data":"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"}
Sep 18 20:41:14.439634 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:14.439587    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" event={"ID":"b61a78fe-e7a3-4df3-9de0-2221c42f54d0","Type":"ContainerStarted","Data":"1535b3a58e038238933d474ac19e6fba26fb532993dbdbee4b6ea120e187c64b"}
Sep 18 20:41:14.460283 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:14.460234    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" podStartSLOduration=1.460219285 podStartE2EDuration="1.460219285s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:41:13 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:41:14.458581061 +0000 UTC m=+2418.587883012" watchObservedRunningTime="2026-09-18 20:41:14.460219285 +0000 UTC m=+2418.589521234"
Sep 18 20:41:23.799227 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:23.799190    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:23.799580 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:23.799247    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:23.811364 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:23.811342    2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:24.506719 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:24.506693    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:41:58.744210 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:41:58.744183    2566 scope.go:117] "RemoveContainer" containerID="8ca46e666a162b5ae60c4ca2daea2232e73ae2ebca26204131c2641f36a6985f"
Sep 18 20:42:14.272504 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:14.272471    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:42:29.291203 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.291165    2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" podUID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" containerName="main" containerID="cri-o://c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860" gracePeriod=45
Sep 18 20:42:29.536268 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.536242    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:42:29.603976 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.603891    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.603976 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.603933    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.604233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.603987    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.604233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604023    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-49j6t\" (UniqueName: \"kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.604233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604044    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.604233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604094    2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir\") pod \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\" (UID: \"b61a78fe-e7a3-4df3-9de0-2221c42f54d0\") "
Sep 18 20:42:29.604233 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604207    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home" (OuterVolumeSpecName: "home") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:29.604462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604345    2566 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-home\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.604462 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604433    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:29.604528 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.604469    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:29.606132 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.606112    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm" (OuterVolumeSpecName: "dshm") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 18 20:42:29.606553 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.606531    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 18 20:42:29.606621 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.606537    2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t" (OuterVolumeSpecName: "kube-api-access-49j6t") pod "b61a78fe-e7a3-4df3-9de0-2221c42f54d0" (UID: "b61a78fe-e7a3-4df3-9de0-2221c42f54d0"). InnerVolumeSpecName "kube-api-access-49j6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 18 20:42:29.705853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.705818    2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-49j6t\" (UniqueName: \"kubernetes.io/projected/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-kube-api-access-49j6t\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.705853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.705846    2566 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-model-cache\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.705853 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.705858    2566 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tmp-dir\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.706098 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.705869    2566 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-tls-certs\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.706098 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.705881    2566 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b61a78fe-e7a3-4df3-9de0-2221c42f54d0-dshm\") on node \"ip-10-0-131-20.ec2.internal\" DevicePath \"\""
Sep 18 20:42:29.838786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.838752    2566 generic.go:356] "Generic (PLEG): container finished" podID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" containerID="c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860" exitCode=0
Sep 18 20:42:29.838930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.838797    2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"
Sep 18 20:42:29.838930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.838840    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" event={"ID":"b61a78fe-e7a3-4df3-9de0-2221c42f54d0","Type":"ContainerDied","Data":"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"}
Sep 18 20:42:29.838930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.838882    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59" event={"ID":"b61a78fe-e7a3-4df3-9de0-2221c42f54d0","Type":"ContainerDied","Data":"1535b3a58e038238933d474ac19e6fba26fb532993dbdbee4b6ea120e187c64b"}
Sep 18 20:42:29.838930 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.838899    2566 scope.go:117] "RemoveContainer" containerID="c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"
Sep 18 20:42:29.848193 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.848175    2566 scope.go:117] "RemoveContainer" containerID="c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"
Sep 18 20:42:29.848441 ip-10-0-131-20 kubenswrapper[2566]: E0918 20:42:29.848424    2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860\": container with ID starting with c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860 not found: ID does not exist" containerID="c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"
Sep 18 20:42:29.848507 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.848447    2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860"} err="failed to get container status \"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860\": rpc error: code = NotFound desc = could not find container \"c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860\": container with ID starting with c585ac396539724b2b55dd00fc1f626d0b701be65629982aadf5adbeb0ce4860 not found: ID does not exist"
Sep 18 20:42:29.859511 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.859449    2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:42:29.865297 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:29.865270    2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-a-7e57cbca-kserve-57b5f5b98c-cvb59"]
Sep 18 20:42:30.080390 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:30.080356    2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" path="/var/lib/kubelet/pods/b61a78fe-e7a3-4df3-9de0-2221c42f54d0/volumes"
Sep 18 20:42:58.764207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:58.764178    2566 scope.go:117] "RemoveContainer" containerID="49b84d6d7e3e786d194a5d0c48a3f3418c06a41b486a93ebf8b053aa0c1d8fe0"
Sep 18 20:42:58.773262 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:42:58.773232    2566 scope.go:117] "RemoveContainer" containerID="84672156bc58d0ebdb64e2dea9b36e8f0fe5b89562f317cbcf04b3e046c10a52"
Sep 18 20:43:00.063113 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.063075    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwttx/must-gather-2n2vl"]
Sep 18 20:43:00.063600 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.063456    2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" containerName="main"
Sep 18 20:43:00.063600 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.063467    2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" containerName="main"
Sep 18 20:43:00.063600 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.063535    2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="b61a78fe-e7a3-4df3-9de0-2221c42f54d0" containerName="main"
Sep 18 20:43:00.068485 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.068465    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.070822 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.070802    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwttx\"/\"kube-root-ca.crt\""
Sep 18 20:43:00.070917 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.070836    2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-jwttx\"/\"default-dockercfg-hp79t\""
Sep 18 20:43:00.070917 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.070810    2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-jwttx\"/\"openshift-service-ca.crt\""
Sep 18 20:43:00.081552 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.081533    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/must-gather-2n2vl"]
Sep 18 20:43:00.085352 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.085328    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-must-gather-output\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.085439 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.085423    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-892nx\" (UniqueName: \"kubernetes.io/projected/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-kube-api-access-892nx\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.186021 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.185997    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-892nx\" (UniqueName: \"kubernetes.io/projected/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-kube-api-access-892nx\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.186134 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.186108    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-must-gather-output\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.186356 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.186341    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-must-gather-output\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.197493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.197474    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-892nx\" (UniqueName: \"kubernetes.io/projected/2dfc0829-834e-402b-bc7b-c94d5ed9f6c9-kube-api-access-892nx\") pod \"must-gather-2n2vl\" (UID: \"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9\") " pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.379883 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.379806    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/must-gather-2n2vl"
Sep 18 20:43:00.648449 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.648419    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/must-gather-2n2vl"]
Sep 18 20:43:00.650445 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:43:00.650411    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2dfc0829_834e_402b_bc7b_c94d5ed9f6c9.slice/crio-e6b7fe779824ca1f08936f8649631fb12dd43cc8e273d8a788a25224bf0db55d WatchSource:0}: Error finding container e6b7fe779824ca1f08936f8649631fb12dd43cc8e273d8a788a25224bf0db55d: Status 404 returned error can't find the container with id e6b7fe779824ca1f08936f8649631fb12dd43cc8e273d8a788a25224bf0db55d
Sep 18 20:43:00.652712 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:00.652695    2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 18 20:43:01.010742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:01.010711    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/must-gather-2n2vl" event={"ID":"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9","Type":"ContainerStarted","Data":"e6b7fe779824ca1f08936f8649631fb12dd43cc8e273d8a788a25224bf0db55d"}
Sep 18 20:43:02.022675 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:02.021819    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/must-gather-2n2vl" event={"ID":"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9","Type":"ContainerStarted","Data":"fe5e03874a8383bb4a20f860a80a187cfb35c006654a4a81c9a719d30dffb9d5"}
Sep 18 20:43:02.022675 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:02.021875    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/must-gather-2n2vl" event={"ID":"2dfc0829-834e-402b-bc7b-c94d5ed9f6c9","Type":"ContainerStarted","Data":"64f61a53a5bf4985ed702e9d83b6201c97123c9be6eb303ed921997e692ed3f0"}
Sep 18 20:43:02.039240 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:02.039183    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwttx/must-gather-2n2vl" podStartSLOduration=1.29270635 podStartE2EDuration="2.039159889s" totalImagesPullingTime="746.453539ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-18 20:43:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:43:02.036804106 +0000 UTC m=+2526.166106057" watchObservedRunningTime="2026-09-18 20:43:02.039159889 +0000 UTC m=+2526.168461841"
Sep 18 20:43:02.944003 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:02.943972    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-zbm8h_1abab961-0419-4eb6-b08f-78d713072033/global-pull-secret-syncer/0.log"
Sep 18 20:43:03.030097 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:03.030050    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-mvnvn_85665792-b3e0-4b6a-b7f2-cab42e82cd62/konnectivity-agent/0.log"
Sep 18 20:43:03.200951 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:03.200875    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-131-20.ec2.internal_857cbacc60cccfe8f3eccb86ceb6ba23/haproxy/0.log"
Sep 18 20:43:07.573583 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:07.573549    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-4s2rd_4c098713-8210-44ee-aefd-5d523861830b/manager/0.log"
Sep 18 20:43:07.641978 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:07.641928    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-5769f99d9f-hcj4p_69416089-53df-4ad0-8fea-bce2b992a351/manager/0.log"
Sep 18 20:43:09.458185 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.458146    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-mj9nm_659002d6-e06b-4bce-9329-61b69116bb31/kube-state-metrics/0.log"
Sep 18 20:43:09.475961 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.475921    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-mj9nm_659002d6-e06b-4bce-9329-61b69116bb31/kube-rbac-proxy-main/0.log"
Sep 18 20:43:09.498107 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.498076    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-84c885658c-mj9nm_659002d6-e06b-4bce-9329-61b69116bb31/kube-rbac-proxy-self/0.log"
Sep 18 20:43:09.654987 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.654945    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5fhs_c2993b00-deeb-400f-9c59-306ba1adc201/node-exporter/0.log"
Sep 18 20:43:09.674892 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.674815    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5fhs_c2993b00-deeb-400f-9c59-306ba1adc201/kube-rbac-proxy/0.log"
Sep 18 20:43:09.694787 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.694755    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-m5fhs_c2993b00-deeb-400f-9c59-306ba1adc201/init-textfile/0.log"
Sep 18 20:43:09.853786 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.853753    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rjc2l_6dc86046-9015-44f6-b8fc-2fad8d868f13/kube-rbac-proxy-main/0.log"
Sep 18 20:43:09.874162 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.874136    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rjc2l_6dc86046-9015-44f6-b8fc-2fad8d868f13/kube-rbac-proxy-self/0.log"
Sep 18 20:43:09.902435 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.902409    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-86c5bc79d6-rjc2l_6dc86046-9015-44f6-b8fc-2fad8d868f13/openshift-state-metrics/0.log"
Sep 18 20:43:09.943993 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.943887    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/prometheus/0.log"
Sep 18 20:43:09.980078 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:09.979228    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/config-reloader/0.log"
Sep 18 20:43:10.016078 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.015507    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/thanos-sidecar/0.log"
Sep 18 20:43:10.045918 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.045894    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/kube-rbac-proxy-web/0.log"
Sep 18 20:43:10.076134 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.076103    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/kube-rbac-proxy/0.log"
Sep 18 20:43:10.128823 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.128707    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/kube-rbac-proxy-thanos/0.log"
Sep 18 20:43:10.152818 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.152793    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_f4c43adc-b596-4d86-8fee-bac0c6cd854c/init-config-reloader/0.log"
Sep 18 20:43:10.190528 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.190501    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-cckpb_4887d16f-9391-419b-9a85-1560af904530/prometheus-operator/0.log"
Sep 18 20:43:10.213967 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:10.213919    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7bd5d56d6-cckpb_4887d16f-9391-419b-9a85-1560af904530/kube-rbac-proxy/0.log"
Sep 18 20:43:11.859510 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:11.859482    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-6446c8d75d-9csd8_c397f480-13a7-4cd9-8dd5-690d317bfb4d/networking-console-plugin/0.log"
Sep 18 20:43:12.658862 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.658824    2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"]
Sep 18 20:43:12.662333 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.662312    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.674785 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.674755    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"]
Sep 18 20:43:12.738202 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.738167    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-558lt\" (UniqueName: \"kubernetes.io/projected/07181c82-bc30-4254-9ca5-5a46e317f094-kube-api-access-558lt\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.738525 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.738503    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-podres\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.738729 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.738712    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-sys\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.738939 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.738924    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-lib-modules\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.739141 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.739124    2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-proc\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.840411 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840304    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-proc\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.840411 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840360    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-558lt\" (UniqueName: \"kubernetes.io/projected/07181c82-bc30-4254-9ca5-5a46e317f094-kube-api-access-558lt\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.840411 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840378    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-podres\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840429    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-sys\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840496    2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-lib-modules\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840705    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-lib-modules\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840806    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-podres\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840852    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-sys\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.840895    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/07181c82-bc30-4254-9ca5-5a46e317f094-proc\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.848427 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.847952    2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-558lt\" (UniqueName: \"kubernetes.io/projected/07181c82-bc30-4254-9ca5-5a46e317f094-kube-api-access-558lt\") pod \"perf-node-gather-daemonset-2z4md\" (UID: \"07181c82-bc30-4254-9ca5-5a46e317f094\") " pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:12.976349 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:12.976310    2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:13.006747 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:13.006705    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-84b44d47-7ckhq_79cdcc90-94be-419e-ac93-1c4122a6e0cb/console/0.log"
Sep 18 20:43:13.255616 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:13.255444    2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"]
Sep 18 20:43:13.257904 ip-10-0-131-20 kubenswrapper[2566]: W0918 20:43:13.257875    2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod07181c82_bc30_4254_9ca5_5a46e317f094.slice/crio-a83ddaf5742d60f4bff5566b9351327ddb2284a63b6ffdf67f16b54248d3278f WatchSource:0}: Error finding container a83ddaf5742d60f4bff5566b9351327ddb2284a63b6ffdf67f16b54248d3278f: Status 404 returned error can't find the container with id a83ddaf5742d60f4bff5566b9351327ddb2284a63b6ffdf67f16b54248d3278f
Sep 18 20:43:13.692336 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:13.692269    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5c4db8bdf9-rnwzn_78a62a93-f02f-4931-ac53-4e6f63f0d96d/volume-data-source-validator/0.log"
Sep 18 20:43:14.141810 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.141767    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md" event={"ID":"07181c82-bc30-4254-9ca5-5a46e317f094","Type":"ContainerStarted","Data":"cd846d3e60a7c37f0ddd2b3a8406a475aec7be0cbd003f156746e3290fc90c01"}
Sep 18 20:43:14.142321 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.141837    2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md" event={"ID":"07181c82-bc30-4254-9ca5-5a46e317f094","Type":"ContainerStarted","Data":"a83ddaf5742d60f4bff5566b9351327ddb2284a63b6ffdf67f16b54248d3278f"}
Sep 18 20:43:14.142730 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.142704    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:14.169266 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.169216    2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md" podStartSLOduration=2.169196306 podStartE2EDuration="2.169196306s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-18 20:43:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-18 20:43:14.167831343 +0000 UTC m=+2538.297133292" watchObservedRunningTime="2026-09-18 20:43:14.169196306 +0000 UTC m=+2538.298498256"
Sep 18 20:43:14.853705 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.853676    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qvdjd_bd5be920-996d-492f-8de2-83ab8c7b1d30/dns/0.log"
Sep 18 20:43:14.872522 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.872487    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qvdjd_bd5be920-996d-492f-8de2-83ab8c7b1d30/kube-rbac-proxy/0.log"
Sep 18 20:43:14.894207 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:14.894181    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-67q5n_b69fb34b-9443-4cea-9b7c-19ba8258730a/dns-node-resolver/0.log"
Sep 18 20:43:15.552765 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:15.552728    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7h7pk_5c0aac5c-761f-4bf0-a9fc-ed9dd9c86755/node-ca/0.log"
Sep 18 20:43:16.500002 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:16.499968    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x_9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2/istio-proxy/0.log"
Sep 18 20:43:16.913099 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:16.913011    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/0.log"
Sep 18 20:43:16.917790 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:16.917767    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/1.log"
Sep 18 20:43:16.936758 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:16.936731    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-rwgf8_896a9164-73dd-44bb-81ea-e8ea10f76ce6/payload-pre-processing/0.log"
Sep 18 20:43:16.988340 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:16.988309    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-94896f94d-jhtw4_4b3df289-28e4-4f05-a9be-fc6f09f22ef6/router/0.log"
Sep 18 20:43:17.892047 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:17.892019    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-zrnwv_a6e4fdd6-d434-4c3f-af0b-0f7531647a85/serve-healthcheck-canary/0.log"
Sep 18 20:43:18.341707 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:18.341680    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-2pxgc_5b564ccd-d8f3-4d3a-91f6-bebb2043fccb/insights-operator/0.log"
Sep 18 20:43:18.342314 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:18.341971    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-6f9c644dd9-2pxgc_5b564ccd-d8f3-4d3a-91f6-bebb2043fccb/insights-operator/1.log"
Sep 18 20:43:18.428017 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:18.427987    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cctw5_9e3eee4c-92e3-4095-b0e6-d00eb6893fa3/kube-rbac-proxy/0.log"
Sep 18 20:43:18.448368 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:18.448340    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cctw5_9e3eee4c-92e3-4095-b0e6-d00eb6893fa3/exporter/0.log"
Sep 18 20:43:18.469637 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:18.469611    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-cctw5_9e3eee4c-92e3-4095-b0e6-d00eb6893fa3/extractor/0.log"
Sep 18 20:43:20.543271 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:20.543235    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-2ctb4_2be4f5eb-ab90-4b6d-b7a9-4e147f52be11/manager/0.log"
Sep 18 20:43:20.955668 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:20.955642    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-728vz_d260a66a-c621-4346-aaf0-e3b0f5374480/manager/0.log"
Sep 18 20:43:21.161994 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:21.161959    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-668c7cd96d-98279_68d37f52-8f5c-4c28-9f08-ba731575d78d/manager/0.log"
Sep 18 20:43:21.162504 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:21.162480    2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-jwttx/perf-node-gather-daemonset-2z4md"
Sep 18 20:43:21.186854 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:21.186827    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-668c7cd96d-98279_68d37f52-8f5c-4c28-9f08-ba731575d78d/copy-manifests/0.log"
Sep 18 20:43:22.535385 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:22.535328    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-76d6fbff56-zrxqm_0ae879f3-7285-44eb-8bac-6c2549860955/manager/0.log"
Sep 18 20:43:22.553282 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:22.553199    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x_9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2/istio-proxy/0.log"
Sep 18 20:43:22.961595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:22.959785    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/0.log"
Sep 18 20:43:22.961595 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:22.961547    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/1.log"
Sep 18 20:43:22.974776 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:22.974749    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-rwgf8_896a9164-73dd-44bb-81ea-e8ea10f76ce6/payload-pre-processing/0.log"
Sep 18 20:43:23.017396 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:23.017363    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-94896f94d-jhtw4_4b3df289-28e4-4f05-a9be-fc6f09f22ef6/router/0.log"
Sep 18 20:43:24.535923 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:24.535891    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-d66cd8c7c-9845x_9f7a7cbc-36de-48bb-9b53-7f6f0c1e8ec2/istio-proxy/0.log"
Sep 18 20:43:24.945554 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:24.945516    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/0.log"
Sep 18 20:43:24.950581 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:24.950546    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-tt5l7_7180073a-a6e8-4fa2-9bce-cba93c382bb1/istio-proxy/1.log"
Sep 18 20:43:24.959545 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:24.959521    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-76f657cdc4-rwgf8_896a9164-73dd-44bb-81ea-e8ea10f76ce6/payload-pre-processing/0.log"
Sep 18 20:43:24.993692 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:24.993669    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-94896f94d-jhtw4_4b3df289-28e4-4f05-a9be-fc6f09f22ef6/router/0.log"
Sep 18 20:43:27.584493 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:27.584466    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-hczmk_0c470b42-45f0-464f-8714-de32ee575948/migrator/0.log"
Sep 18 20:43:27.605789 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:27.605759    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-76794cbd6c-hczmk_0c470b42-45f0-464f-8714-de32ee575948/graceful-termination/0.log"
Sep 18 20:43:27.948742 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:27.948660    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-k6krk_e39a38af-479b-4c64-91d2-70f12b9525b4/kube-storage-version-migrator-operator/1.log"
Sep 18 20:43:27.950253 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:27.950223    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-55889f7b47-k6krk_e39a38af-479b-4c64-91d2-70f12b9525b4/kube-storage-version-migrator-operator/0.log"
Sep 18 20:43:28.914215 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:28.914189    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-79qgh_c367a973-1121-4c33-895f-e2b2f4e60b0f/kube-multus/0.log"
Sep 18 20:43:28.936937 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:28.936917    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/kube-multus-additional-cni-plugins/0.log"
Sep 18 20:43:28.958132 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:28.958113    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/egress-router-binary-copy/0.log"
Sep 18 20:43:28.980947 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:28.980927    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/cni-plugins/0.log"
Sep 18 20:43:29.002136 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.002115    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/bond-cni-plugin/0.log"
Sep 18 20:43:29.022428 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.022410    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/routeoverride-cni/0.log"
Sep 18 20:43:29.042946 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.042920    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/whereabouts-cni-bincopy/0.log"
Sep 18 20:43:29.062589 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.062565    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-7r7ct_e05ba3d9-56f2-4a3b-89b2-4511ace2afbf/whereabouts-cni/0.log"
Sep 18 20:43:29.439574 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.439546    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4jr7x_d7a6ca69-b215-437a-9191-d46540f76181/network-metrics-daemon/0.log"
Sep 18 20:43:29.457984 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:29.457959    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-4jr7x_d7a6ca69-b215-437a-9191-d46540f76181/kube-rbac-proxy/0.log"
Sep 18 20:43:30.303657 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.303627    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-controller/0.log"
Sep 18 20:43:30.320330 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.320308    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/1.log"
Sep 18 20:43:30.320433 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.320373    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovn-acl-logging/0.log"
Sep 18 20:43:30.345127 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.345105    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/kube-rbac-proxy-node/0.log"
Sep 18 20:43:30.369662 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.369637    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/kube-rbac-proxy-ovn-metrics/0.log"
Sep 18 20:43:30.386677 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.386659    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/northd/0.log"
Sep 18 20:43:30.407016 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.407000    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/nbdb/0.log"
Sep 18 20:43:30.427195 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.427177    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/sbdb/0.log"
Sep 18 20:43:30.529970 ip-10-0-131-20 kubenswrapper[2566]: I0918 20:43:30.529951    2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-5xrvx_e10a48b8-18a8-415e-8240-5072512f4f1f/ovnkube-controller/0.log"