Sep 11 13:02:37.573076 ip-10-0-142-204 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
Sep 11 13:02:37.573087 ip-10-0-142-204 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
Sep 11 13:02:37.573093 ip-10-0-142-204 systemd[1]: kubelet.service: Failed with result 'resources'.
Sep 11 13:02:37.573228 ip-10-0-142-204 systemd[1]: Failed to start Kubernetes Kubelet.
Sep 11 13:02:47.734496 ip-10-0-142-204 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
Sep 11 13:02:47.734515 ip-10-0-142-204 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot 443904c5018e4118927924ba9bf935f7 --
Sep 11 13:04:45.368011 ip-10-0-142-204 systemd[1]: Starting Kubernetes Kubelet...
Sep 11 13:04:45.737782 ip-10-0-142-204 kubenswrapper[2573]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:45.737782 ip-10-0-142-204 kubenswrapper[2573]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
Sep 11 13:04:45.737782 ip-10-0-142-204 kubenswrapper[2573]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:45.737782 ip-10-0-142-204 kubenswrapper[2573]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
Sep 11 13:04:45.737782 ip-10-0-142-204 kubenswrapper[2573]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
Sep 11 13:04:45.740175 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.740084    2573 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
Sep 11 13:04:45.744640 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744621    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.744640 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744638    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.744640 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744642    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744646    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744650    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744654    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744657    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744660    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744664    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744667    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744670    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744673    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744676    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744679    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744682    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744684    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744688    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744691    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744693    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744696    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744698    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744701    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.744748 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744715    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744718    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744721    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744723    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744726    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744729    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744731    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744734    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744739    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744742    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744746    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744749    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744752    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744755    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744758    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744760    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744763    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744766    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744768    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.745210 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744771    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744774    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744778    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744780    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744783    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744785    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744788    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744790    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744794    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744796    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744799    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744801    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744803    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744806    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744808    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744812    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744814    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744817    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744819    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.745691 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744822    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744824    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744827    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744830    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744832    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744835    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744837    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744840    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744843    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744845    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744847    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744850    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744853    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744855    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744858    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744860    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744863    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744865    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744868    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744870    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.746171 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744873    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744875    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744880    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744883    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744886    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744888    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744890    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744893    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744897    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744900    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744903    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744905    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744908    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744911    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744913    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744916    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744918    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744921    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.744925    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.746654 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745613    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745620    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745623    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745626    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745630    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745632    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745635    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745638    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745640    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745643    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745645    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745648    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745650    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745652    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745655    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745657    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745661    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745663    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745666    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.747131 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745668    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745671    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745675    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745677    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745680    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745683    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745686    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745688    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745691    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745693    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745695    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745698    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745701    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745717    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745720    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745722    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745725    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745729    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745733    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.747588 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745735    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745738    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745741    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745744    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745746    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745748    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745751    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745753    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745756    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745758    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745760    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745764    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745768    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745770    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745773    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745777    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745781    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745784    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745787    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.748062 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745789    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745792    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745795    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745797    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745799    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745802    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745804    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745807    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745809    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745812    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745816    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745818    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745820    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745823    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745826    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745840    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745844    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745847    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745851    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745854    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.748513 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745856    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745859    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745862    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745864    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745867    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745870    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745873    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745876    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745879    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745882    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745885    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745887    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745890    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745892    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745895    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745897    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745900    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745902    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745904    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745907    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.749003 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745909    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.745912    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746286    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746291    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746294    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746296    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746299    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746302    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746305    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746308    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746310    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746313    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746315    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746318    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746320    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746323    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746325    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746329    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746332    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746335    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.749475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746337    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746340    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746342    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746345    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746348    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746352    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746355    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746358    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746361    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746363    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746365    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746368    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746370    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746373    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746375    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746377    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746380    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746382    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746385    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.749982 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746388    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746390    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746393    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746395    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746398    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746400    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746402    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746405    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746407    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746409    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746414    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746417    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746420    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746422    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746426    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746429    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746432    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746435    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746437    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.750437 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746440    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746442    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746444    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746447    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746449    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746452    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746454    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746457    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746460    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746462    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746465    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746468    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746470    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746473    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746475    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746478    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746480    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746483    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746485    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746489    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.750922 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746492    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746495    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746498    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746502    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746504    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746507    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746510    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746512    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746514    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746517    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746520    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746523    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746525    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746528    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746530    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746533    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746535    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746537    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746540    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746542    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.751404 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746545    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746547    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.746550    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746629    2573 flags.go:64] FLAG: --address="0.0.0.0"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746663    2573 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746670    2573 flags.go:64] FLAG: --anonymous-auth="true"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746675    2573 flags.go:64] FLAG: --application-metrics-count-limit="100"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746679    2573 flags.go:64] FLAG: --authentication-token-webhook="false"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746683    2573 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746687    2573 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746691    2573 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746694    2573 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746698    2573 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746701    2573 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746716    2573 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746721    2573 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746724    2573 flags.go:64] FLAG: --cgroup-root=""
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746726    2573 flags.go:64] FLAG: --cgroups-per-qos="true"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746731    2573 flags.go:64] FLAG: --client-ca-file=""
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746733    2573 flags.go:64] FLAG: --cloud-provider="external"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746736    2573 flags.go:64] FLAG: --cluster-dns="[]"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746740    2573 flags.go:64] FLAG: --cluster-domain=""
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746743    2573 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746746    2573 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d"
Sep 11 13:04:45.751901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746749    2573 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746753    2573 flags.go:64] FLAG: --container-log-max-files="5"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746756    2573 flags.go:64] FLAG: --container-log-max-size="10Mi"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746759    2573 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746762    2573 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746766    2573 flags.go:64] FLAG: --containerd-namespace="k8s.io"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746768    2573 flags.go:64] FLAG: --contention-profiling="false"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746771    2573 flags.go:64] FLAG: --cpu-cfs-quota="true"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746774    2573 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746777    2573 flags.go:64] FLAG: --cpu-manager-policy="none"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746780    2573 flags.go:64] FLAG: --cpu-manager-policy-options=""
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746786    2573 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746789    2573 flags.go:64] FLAG: --enable-controller-attach-detach="true"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746792    2573 flags.go:64] FLAG: --enable-debugging-handlers="true"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746795    2573 flags.go:64] FLAG: --enable-load-reader="false"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746798    2573 flags.go:64] FLAG: --enable-server="true"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746801    2573 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746806    2573 flags.go:64] FLAG: --event-burst="100"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746809    2573 flags.go:64] FLAG: --event-qps="50"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746812    2573 flags.go:64] FLAG: --event-storage-age-limit="default=0"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746815    2573 flags.go:64] FLAG: --event-storage-event-limit="default=0"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746818    2573 flags.go:64] FLAG: --eviction-hard=""
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746821    2573 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746825    2573 flags.go:64] FLAG: --eviction-minimum-reclaim=""
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746828    2573 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
Sep 11 13:04:45.752581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746831    2573 flags.go:64] FLAG: --eviction-soft=""
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746834    2573 flags.go:64] FLAG: --eviction-soft-grace-period=""
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746837    2573 flags.go:64] FLAG: --exit-on-lock-contention="false"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746841    2573 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746844    2573 flags.go:64] FLAG: --experimental-mounter-path=""
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746847    2573 flags.go:64] FLAG: --fail-cgroupv1="false"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746849    2573 flags.go:64] FLAG: --fail-swap-on="true"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746852    2573 flags.go:64] FLAG: --feature-gates=""
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746856    2573 flags.go:64] FLAG: --file-check-frequency="20s"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746860    2573 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746863    2573 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746866    2573 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746869    2573 flags.go:64] FLAG: --healthz-port="10248"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746872    2573 flags.go:64] FLAG: --help="false"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746875    2573 flags.go:64] FLAG: --hostname-override="ip-10-0-142-204.ec2.internal"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746878    2573 flags.go:64] FLAG: --housekeeping-interval="10s"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746881    2573 flags.go:64] FLAG: --http-check-frequency="20s"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746883    2573 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746889    2573 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746893    2573 flags.go:64] FLAG: --image-gc-high-threshold="85"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746895    2573 flags.go:64] FLAG: --image-gc-low-threshold="80"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746898    2573 flags.go:64] FLAG: --image-service-endpoint=""
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746901    2573 flags.go:64] FLAG: --kernel-memcg-notification="false"
Sep 11 13:04:45.753182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746904    2573 flags.go:64] FLAG: --kube-api-burst="100"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746907    2573 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746911    2573 flags.go:64] FLAG: --kube-api-qps="50"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746913    2573 flags.go:64] FLAG: --kube-reserved=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746916    2573 flags.go:64] FLAG: --kube-reserved-cgroup=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746919    2573 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746923    2573 flags.go:64] FLAG: --kubelet-cgroups=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746925    2573 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746928    2573 flags.go:64] FLAG: --lock-file=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746931    2573 flags.go:64] FLAG: --log-cadvisor-usage="false"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746934    2573 flags.go:64] FLAG: --log-flush-frequency="5s"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746937    2573 flags.go:64] FLAG: --log-json-info-buffer-size="0"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746943    2573 flags.go:64] FLAG: --log-json-split-stream="false"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746945    2573 flags.go:64] FLAG: --log-text-info-buffer-size="0"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746948    2573 flags.go:64] FLAG: --log-text-split-stream="false"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746951    2573 flags.go:64] FLAG: --logging-format="text"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746954    2573 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746958    2573 flags.go:64] FLAG: --make-iptables-util-chains="true"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746961    2573 flags.go:64] FLAG: --manifest-url=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746964    2573 flags.go:64] FLAG: --manifest-url-header=""
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746968    2573 flags.go:64] FLAG: --max-housekeeping-interval="15s"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746971    2573 flags.go:64] FLAG: --max-open-files="1000000"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746975    2573 flags.go:64] FLAG: --max-pods="110"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746978    2573 flags.go:64] FLAG: --maximum-dead-containers="-1"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746981    2573 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
Sep 11 13:04:45.753774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746983    2573 flags.go:64] FLAG: --memory-manager-policy="None"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746986    2573 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746991    2573 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746995    2573 flags.go:64] FLAG: --node-ip="0.0.0.0"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.746998    2573 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747004    2573 flags.go:64] FLAG: --node-status-max-images="50"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747007    2573 flags.go:64] FLAG: --node-status-update-frequency="10s"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747010    2573 flags.go:64] FLAG: --oom-score-adj="-999"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747013    2573 flags.go:64] FLAG: --pod-cidr=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747016    2573 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:eee392dbe3ac2ed4a38bca29ae3bd486f5b7780406c62c600154cd1906871b10"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747021    2573 flags.go:64] FLAG: --pod-manifest-path=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747024    2573 flags.go:64] FLAG: --pod-max-pids="-1"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747027    2573 flags.go:64] FLAG: --pods-per-core="0"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747029    2573 flags.go:64] FLAG: --port="10250"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747032    2573 flags.go:64] FLAG: --protect-kernel-defaults="false"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747035    2573 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-05c92effffb1f6c29"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747038    2573 flags.go:64] FLAG: --qos-reserved=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747041    2573 flags.go:64] FLAG: --read-only-port="10255"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747044    2573 flags.go:64] FLAG: --register-node="true"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747046    2573 flags.go:64] FLAG: --register-with-taints=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747051    2573 flags.go:64] FLAG: --registry-burst="10"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747053    2573 flags.go:64] FLAG: --registry-qps="5"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747056    2573 flags.go:64] FLAG: --reserved-cpus=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747059    2573 flags.go:64] FLAG: --reserved-memory=""
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747062    2573 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
Sep 11 13:04:45.754466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747065    2573 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747068    2573 flags.go:64] FLAG: --rotate-certificates="false"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747071    2573 flags.go:64] FLAG: --rotate-server-certificates="false"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747073    2573 flags.go:64] FLAG: --runonce="false"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747076    2573 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747079    2573 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747082    2573 flags.go:64] FLAG: --seccomp-default="false"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747085    2573 flags.go:64] FLAG: --serialize-image-pulls="true"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747087    2573 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747091    2573 flags.go:64] FLAG: --storage-driver-db="cadvisor"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747095    2573 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747098    2573 flags.go:64] FLAG: --storage-driver-password="root"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747101    2573 flags.go:64] FLAG: --storage-driver-secure="false"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747104    2573 flags.go:64] FLAG: --storage-driver-table="stats"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747107    2573 flags.go:64] FLAG: --storage-driver-user="root"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747109    2573 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747112    2573 flags.go:64] FLAG: --sync-frequency="1m0s"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747115    2573 flags.go:64] FLAG: --system-cgroups=""
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747118    2573 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747123    2573 flags.go:64] FLAG: --system-reserved-cgroup=""
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747126    2573 flags.go:64] FLAG: --tls-cert-file=""
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747129    2573 flags.go:64] FLAG: --tls-cipher-suites="[]"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747133    2573 flags.go:64] FLAG: --tls-min-version=""
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747136    2573 flags.go:64] FLAG: --tls-private-key-file=""
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747139    2573 flags.go:64] FLAG: --topology-manager-policy="none"
Sep 11 13:04:45.755403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747142    2573 flags.go:64] FLAG: --topology-manager-policy-options=""
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747144    2573 flags.go:64] FLAG: --topology-manager-scope="container"
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747147    2573 flags.go:64] FLAG: --v="2"
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747151    2573 flags.go:64] FLAG: --version="false"
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747155    2573 flags.go:64] FLAG: --vmodule=""
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747159    2573 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.747162    2573 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747275    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747278    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747281    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747283    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747286    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747289    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747292    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747295    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747297    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747302    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747305    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747308    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747311    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747313    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.756034 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747316    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747319    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747321    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747324    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747327    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747329    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747332    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747334    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747337    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747339    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747342    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747345    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747347    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747350    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747352    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747355    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747358    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747361    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747364    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.756572 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747375    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747378    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747381    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747383    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747386    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747388    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747391    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747395    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747399    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747403    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747406    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747409    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747412    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747415    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747417    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747420    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747423    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747426    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747428    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747431    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.757046 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747434    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747436    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747439    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747441    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747444    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747446    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747449    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747451    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747454    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747457    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747459    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747462    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747465    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747467    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747470    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747472    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747475    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747477    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747479    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747482    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.757522 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747486    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747490    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747492    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747495    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747498    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747500    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747503    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747506    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747508    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747511    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747513    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747516    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747518    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747521    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747523    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747527    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747531    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747533    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747536    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747539    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.758019 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747541    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747543    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747546    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747549    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747551    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.747554    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.748220    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.756953    2573 server.go:529] "Kubelet version" kubeletVersion="v1.34.9"
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.756971    2573 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.757870    2573 watchdog_linux.go:95] "Systemd watchdog is not enabled"
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.757878    2573 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757938    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757943    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757946    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.758500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757949    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757952    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757955    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757958    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757961    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757963    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757966    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757969    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757972    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757974    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757977    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757980    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757983    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757990    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757993    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.757997    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758001    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758004    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758007    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.758870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758010    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758013    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758016    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758019    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758022    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758025    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758028    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758032    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758035    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758038    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758041    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758044    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758047    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758050    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758052    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758055    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758057    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758060    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758062    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.759326 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758065    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758067    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758070    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758074    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758077    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758080    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758083    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758086    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758097    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758100    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758103    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758106    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758108    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758111    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758113    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758117    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758119    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758122    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758125    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758127    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.759828 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758131    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758133    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758136    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758138    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758141    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758144    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758146    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758149    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758152    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758155    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758157    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758159    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758162    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758164    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758167    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758171    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758174    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758177    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758180    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758183    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.760310 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758186    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758188    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758191    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758193    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758196    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758198    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758201    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758204    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758206    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758209    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758212    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758214    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758217    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758220    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758222    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758225    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758227    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.760799 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758230    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.758234    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758357    2573 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758363    2573 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758368    2573 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758371    2573 feature_gate.go:328] unrecognized feature gate: OVNObservability
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758374    2573 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758377    2573 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758380    2573 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758383    2573 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758386    2573 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758388    2573 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758391    2573 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758393    2573 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
Sep 11 13:04:45.761200 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758397    2573 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758399    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758402    2573 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758404    2573 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758407    2573 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758409    2573 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758412    2573 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758414    2573 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758416    2573 feature_gate.go:328] unrecognized feature gate: Example
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758419    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758422    2573 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758425    2573 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758428    2573 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758430    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758433    2573 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758435    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758438    2573 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758440    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758443    2573 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758445    2573 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
Sep 11 13:04:45.761539 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758448    2573 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758450    2573 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758453    2573 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758455    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758458    2573 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758460    2573 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758462    2573 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758465    2573 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758467    2573 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758470    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758472    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758474    2573 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758477    2573 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758481    2573 feature_gate.go:328] unrecognized feature gate: EventTTL
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758483    2573 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758486    2573 feature_gate.go:328] unrecognized feature gate: KMSEncryption
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758488    2573 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758491    2573 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758493    2573 feature_gate.go:328] unrecognized feature gate: InsightsConfig
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758495    2573 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
Sep 11 13:04:45.762133 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758498    2573 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758501    2573 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758504    2573 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758507    2573 feature_gate.go:328] unrecognized feature gate: PinnedImages
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758510    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758512    2573 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758516    2573 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release.
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758519    2573 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758522    2573 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758525    2573 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758528    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758531    2573 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758534    2573 feature_gate.go:328] unrecognized feature gate: DualReplica
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758536    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758539    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPI
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758541    2573 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758544    2573 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758546    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758549    2573 feature_gate.go:328] unrecognized feature gate: NewOLM
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758551    2573 feature_gate.go:328] unrecognized feature gate: SignatureStores
Sep 11 13:04:45.762606 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758554    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758556    2573 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758559    2573 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758561    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758564    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758566    2573 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758569    2573 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758572    2573 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758574    2573 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758577    2573 feature_gate.go:328] unrecognized feature gate: OSStreams
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758580    2573 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758582    2573 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758585    2573 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758587    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758590    2573 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758593    2573 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758596    2573 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758598    2573 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758600    2573 feature_gate.go:328] unrecognized feature gate: Example2
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758603    2573 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
Sep 11 13:04:45.763088 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758605    2573 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758608    2573 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758610    2573 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758612    2573 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758615    2573 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758617    2573 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:45.758620    2573 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.758625    2573 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]}
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.758760    2573 server.go:962] "Client rotation is on, will bootstrap in background"
Sep 11 13:04:45.763558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.763143    2573 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
Sep 11 13:04:45.764160 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.764147    2573 server.go:1019] "Starting client certificate rotation"
Sep 11 13:04:45.764262 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.764245    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:45.764294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.764280    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
Sep 11 13:04:45.766463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.766450    2573 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:45.767612 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.767596    2573 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
Sep 11 13:04:45.780131 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.780104    2573 log.go:25] "Validated CRI v1 runtime API"
Sep 11 13:04:45.783097 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.783079    2573 log.go:25] "Validated CRI v1 image API"
Sep 11 13:04:45.783950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.783930    2573 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
Sep 11 13:04:45.786953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.786927    2573 fs.go:135] Filesystem UUIDs: map[2996cede-e18a-4458-83cc-12014e836766:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 ebf4ca72-3e33-4662-878f-ab2aa4792300:/dev/nvme0n1p3]
Sep 11 13:04:45.787023 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.786952    2573 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
Sep 11 13:04:45.793736 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.793610    2573 manager.go:216] Machine: {Timestamp:2026-09-11 13:04:45.792374933 +0000 UTC m=+0.312412268 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3196318 MemoryCapacity:33164476416 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0d18249566f151ccee62cedca002 SystemUUID:ec2d0d18-2495-66f1-51cc-ee62cedca002 BootID:443904c5-018e-4118-9279-24ba9bf935f7 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582238208 Type:vfs Inodes:4048398 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6111232 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582238208 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:53:f2:c6:b8:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:53:f2:c6:b8:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:be:6e:10:8c:4e:3a Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164476416 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
Sep 11 13:04:45.793736 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.793722    2573 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
Sep 11 13:04:45.793901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.793807    2573 manager.go:232] Version: {KernelVersion:5.14.0-570.140.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260908-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
Sep 11 13:04:45.794991 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.794961    2573 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
Sep 11 13:04:45.795133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.794991    2573 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-142-204.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
Sep 11 13:04:45.795764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.795755    2573 topology_manager.go:138] "Creating topology manager with none policy"
Sep 11 13:04:45.795796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.795767    2573 container_manager_linux.go:309] "Creating device plugin manager"
Sep 11 13:04:45.795796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.795775    2573 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:45.796428 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.796408    2573 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
Sep 11 13:04:45.796506 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.796430    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:45.797692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.797679    2573 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager"
Sep 11 13:04:45.801219 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801205    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:45.801349 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801338    2573 server.go:1240] "Using root directory" path="/var/lib/kubelet"
Sep 11 13:04:45.801414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801378    2573 kubelet.go:486] "Attempting to sync node with API server"
Sep 11 13:04:45.801414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801393    2573 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests"
Sep 11 13:04:45.801414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801408    2573 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
Sep 11 13:04:45.801533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801421    2573 kubelet.go:398] "Adding apiserver pod source"
Sep 11 13:04:45.801533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801433    2573 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
Sep 11 13:04:45.801846 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801832    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:45.801909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.801856    2573 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
Sep 11 13:04:45.804068 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.804049    2573 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.12-5.rhaos4.21.gitd1ec63c.el9" apiVersion="v1"
Sep 11 13:04:45.806259 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.806245    2573 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
Sep 11 13:04:45.806308 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.806275    2573 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled"
Sep 11 13:04:45.808486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808473    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
Sep 11 13:04:45.808525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808491    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
Sep 11 13:04:45.808525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808497    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
Sep 11 13:04:45.808525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808502    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
Sep 11 13:04:45.808525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808511    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
Sep 11 13:04:45.808525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808516    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808536    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808541    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808548    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808554    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808562    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected"
Sep 11 13:04:45.808648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.808567    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
Sep 11 13:04:45.809123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.809109    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi"
Sep 11 13:04:45.809152 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.809127    2573 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image"
Sep 11 13:04:45.809303 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.809284    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-142-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
Sep 11 13:04:45.809339 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.809316    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
Sep 11 13:04:45.812321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.812305    2573 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-142-204.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
Sep 11 13:04:45.813063 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.813051    2573 server.go:1268] "Started kubelet"
Sep 11 13:04:45.813154 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.813124    2573 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
Sep 11 13:04:45.813202 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.813136    2573 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
Sep 11 13:04:45.813251 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.813218    2573 server_v1.go:49] "podresources" method="list" useActivePods=true
Sep 11 13:04:45.813913 ip-10-0-142-204 systemd[1]: Started Kubernetes Kubelet.
Sep 11 13:04:45.814682 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.814669    2573 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
Sep 11 13:04:45.815943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.815922    2573 server.go:310] "Adding debug handlers to kubelet server"
Sep 11 13:04:45.819261 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.818992    2573 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
Sep 11 13:04:45.820298 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.819318    2573 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-142-204.ec2.internal.18d445f88e35bb61  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-142-204.ec2.internal,UID:ip-10-0-142-204.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-142-204.ec2.internal,},FirstTimestamp:2026-09-11 13:04:45.813021537 +0000 UTC m=+0.333058872,LastTimestamp:2026-09-11 13:04:45.813021537 +0000 UTC m=+0.333058872,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-142-204.ec2.internal,}"
Sep 11 13:04:45.822287 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.822265    2573 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:45.822375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.822269    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vrk8m"
Sep 11 13:04:45.822841 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.822822    2573 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
Sep 11 13:04:45.822986 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.822970    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:45.823548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823523    2573 desired_state_of_world_populator.go:146] "Desired state populator starts to run"
Sep 11 13:04:45.823548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823526    2573 volume_manager.go:311] "The desired_state_of_world populator starts"
Sep 11 13:04:45.823548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823549    2573 volume_manager.go:313] "Starting Kubelet Volume Manager"
Sep 11 13:04:45.823733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823656    2573 reconstruct.go:97] "Volume reconstruction finished"
Sep 11 13:04:45.823733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823665    2573 reconciler.go:29] "Reconciler: start to sync state"
Sep 11 13:04:45.823830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823785    2573 factory.go:55] Registering systemd factory
Sep 11 13:04:45.823830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823815    2573 factory.go:223] Registration of the systemd container factory successfully
Sep 11 13:04:45.824009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.823994    2573 factory.go:153] Registering CRI-O factory
Sep 11 13:04:45.824009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.824012    2573 factory.go:223] Registration of the crio container factory successfully
Sep 11 13:04:45.824105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.824061    2573 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
Sep 11 13:04:45.824105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.824078    2573 factory.go:103] Registering Raw factory
Sep 11 13:04:45.824105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.824090    2573 manager.go:1196] Started watching for new ooms in manager
Sep 11 13:04:45.824759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.824744    2573 manager.go:349] Starting recovery of all containers
Sep 11 13:04:45.825849 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.825826    2573 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-142-204.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
Sep 11 13:04:45.826404 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.826375    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
Sep 11 13:04:45.829866 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.829838    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-vrk8m"
Sep 11 13:04:45.837650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.837633    2573 manager.go:354] Recovery completed
Sep 11 13:04:45.842538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.842520    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:45.844258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844241    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:45.844333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844268    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:45.844333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844278    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:45.844892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844875    2573 cpu_manager.go:222] "Starting CPU manager" policy="none"
Sep 11 13:04:45.844892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844890    2573 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
Sep 11 13:04:45.845031 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844912    2573 state_mem.go:36] "Initialized new in-memory state store"
Sep 11 13:04:45.845031 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.844891    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:45.845580 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.845566    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:45.845641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.845597    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:45.845641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.845613    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:45.846436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.846318    2573 policy_none.go:49] "None policy: Start"
Sep 11 13:04:45.846436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.846435    2573 memory_manager.go:187] "Starting memorymanager" policy="None"
Sep 11 13:04:45.846579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.846447    2573 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint"
Sep 11 13:04:45.847610 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.847597    2573 policy_none.go:47] "Start"
Sep 11 13:04:45.874370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.874349    2573 manager.go:337] "Starting Device Plugin manager"
Sep 11 13:04:45.874493 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.874398    2573 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
Sep 11 13:04:45.874493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.874411    2573 server.go:87] "Starting device plugin registration server"
Sep 11 13:04:45.874829 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.874815    2573 eviction_manager.go:189] "Eviction manager: starting control loop"
Sep 11 13:04:45.874871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.874833    2573 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
Sep 11 13:04:45.874961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.874941    2573 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
Sep 11 13:04:45.875054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.875008    2573 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
Sep 11 13:04:45.875054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.875016    2573 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
Sep 11 13:04:45.875698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.875682    2573 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
Sep 11 13:04:45.875791 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.875757    2573 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:45.964117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.964075    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4"
Sep 11 13:04:45.965312 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.965292    2573 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6"
Sep 11 13:04:45.965430 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.965348    2573 status_manager.go:244] "Starting to sync pod status with apiserver"
Sep 11 13:04:45.965430 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.965375    2573 kubelet.go:2448] "Starting kubelet main sync loop"
Sep 11 13:04:45.965430 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.965420    2573 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
Sep 11 13:04:45.968303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.968282    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:45.975731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.975712    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:45.976661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.976644    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:45.976740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.976679    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:45.976740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.976690    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:45.976740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.976735    2573 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-142-204.ec2.internal"
Sep 11 13:04:45.985877 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:45.985857    2573 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-142-204.ec2.internal"
Sep 11 13:04:45.985966 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:45.985881    2573 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-142-204.ec2.internal\": node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.006909 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.006831    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.066011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.065984    2573 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"]
Sep 11 13:04:46.066181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.066080    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:46.067686 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.067669    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:46.067784 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.067701    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:46.067784 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.067725    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:46.069015 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069001    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:46.069139 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069124    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.069189 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069154    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:46.069751 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069735    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:46.069834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069743    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:46.069834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069780    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:46.069834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069791    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:46.069834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069761    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:46.069834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.069828    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:46.071197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.071182    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.071266 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.071213    2573 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach"
Sep 11 13:04:46.071893 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.071877    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientMemory"
Sep 11 13:04:46.071972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.071901    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasNoDiskPressure"
Sep 11 13:04:46.071972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.071910    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeHasSufficientPID"
Sep 11 13:04:46.100341 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.100320    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-204.ec2.internal\" not found" node="ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.104741 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.104726    2573 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-142-204.ec2.internal\" not found" node="ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.107754 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.107733    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.126231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.126210    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.126309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.126235    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.126309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.126256    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af658a1e06ce3bdd72fe16298188a4e8-config\") pod \"kube-apiserver-proxy-ip-10-0-142-204.ec2.internal\" (UID: \"af658a1e06ce3bdd72fe16298188a4e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.207825 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.207797    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.226760 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226737    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af658a1e06ce3bdd72fe16298188a4e8-config\") pod \"kube-apiserver-proxy-ip-10-0-142-204.ec2.internal\" (UID: \"af658a1e06ce3bdd72fe16298188a4e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.226830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226766    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.226830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226782    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.226830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226821    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.226916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226825    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/af658a1e06ce3bdd72fe16298188a4e8-config\") pod \"kube-apiserver-proxy-ip-10-0-142-204.ec2.internal\" (UID: \"af658a1e06ce3bdd72fe16298188a4e8\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.226916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.226852    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9f5e10749336ffb5faa04f0c65e06cae-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal\" (UID: \"9f5e10749336ffb5faa04f0c65e06cae\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.308669 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.308599    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.404077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.404047    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.408517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.408281    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.409359 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.409343    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.510381 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.510342    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.612421 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.612385    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.713341 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:46.713302    2573 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-142-204.ec2.internal\" not found"
Sep 11 13:04:46.765796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.765754    2573 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
Sep 11 13:04:46.766305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.765946    2573 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
Sep 11 13:04:46.788087 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.788064    2573 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:46.791075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.789814    2573 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:46.808323 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.808300    2573 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:46.822733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.822700    2573 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
Sep 11 13:04:46.823149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.823129    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.836121 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.836044    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-09-10 12:59:46 +0000 UTC" deadline="2028-02-27 09:23:17.700468269 +0000 UTC"
Sep 11 13:04:46.836121 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.836078    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12812h18m30.864395547s"
Sep 11 13:04:46.836121 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.836101    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:46.838002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.837980    2573 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
Sep 11 13:04:46.838244 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.838226    2573 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal"
Sep 11 13:04:46.845211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.845194    2573 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
Sep 11 13:04:46.863540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.863518    2573 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-c5dpf"
Sep 11 13:04:46.872851 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:46.872828    2573 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-c5dpf"
Sep 11 13:04:47.047424 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:47.047384    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaf658a1e06ce3bdd72fe16298188a4e8.slice/crio-1c7df8816cf64cf53e307ccc7177d158895cb3062f5abf3546a7955d1d226953 WatchSource:0}: Error finding container 1c7df8816cf64cf53e307ccc7177d158895cb3062f5abf3546a7955d1d226953: Status 404 returned error can't find the container with id 1c7df8816cf64cf53e307ccc7177d158895cb3062f5abf3546a7955d1d226953
Sep 11 13:04:47.047741 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:47.047702    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f5e10749336ffb5faa04f0c65e06cae.slice/crio-560a2dc3c3ea1bc615d1bf07cfbda92630364eab23f94ac32012f5f0df41c144 WatchSource:0}: Error finding container 560a2dc3c3ea1bc615d1bf07cfbda92630364eab23f94ac32012f5f0df41c144: Status 404 returned error can't find the container with id 560a2dc3c3ea1bc615d1bf07cfbda92630364eab23f94ac32012f5f0df41c144
Sep 11 13:04:47.054493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.054475    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:04:47.801719 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.801670    2573 apiserver.go:52] "Watching apiserver"
Sep 11 13:04:47.808585 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.808197    2573 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
Sep 11 13:04:47.809609 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.809550    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-nvkr7","openshift-multus/multus-additional-cni-plugins-xp7s6","openshift-multus/network-metrics-daemon-v4ngl","kube-system/konnectivity-agent-j7r7w","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z","openshift-dns/node-resolver-zh8nr","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal","openshift-multus/multus-vxtcq","openshift-network-diagnostics/network-check-target-jz9hm","openshift-network-operator/iptables-alerter-wmvcd","openshift-ovn-kubernetes/ovnkube-node-6lvhv","kube-system/global-pull-secret-syncer-dv8kv","kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal","openshift-cluster-node-tuning-operator/tuned-8fklw"]
Sep 11 13:04:47.811585 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.811556    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.813019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.812989    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.814323 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.814301    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.814434 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.814383    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:47.815054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.815035    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-sj5j4\""
Sep 11 13:04:47.815370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.815353    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.815837 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.815813    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
Sep 11 13:04:47.815930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.815915    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.815997 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.815933    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.816050 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816031    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
Sep 11 13:04:47.816100 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816031    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-dvkdh\""
Sep 11 13:04:47.816198 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816177    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
Sep 11 13:04:47.816265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816219    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
Sep 11 13:04:47.816409 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816391    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.816527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816503    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
Sep 11 13:04:47.816527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.816523    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.819468 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.819442    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.821374 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.821355    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.822536    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.822897    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823008    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823216    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823229    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823460    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823554    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823562    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-shqhz\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.823629    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.823945    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.824171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.824005    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-2vr42\""
Sep 11 13:04:47.830342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.830319    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.831502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.830999    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-wv67q\""
Sep 11 13:04:47.831502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.831472    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.831917 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.831893    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:47.834969 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.834704    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.835216 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835196    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.835353 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835333    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835370    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-klkm7\" (UniqueName: \"kubernetes.io/projected/5367657e-14dc-493e-a64a-5edc6117c72f-kube-api-access-klkm7\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.835478 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/485cb65a-3796-410d-a65a-19dc773993d2-konnectivity-ca\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.835478 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835463    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-conf\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835564 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-etc-tuned\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835564 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835513    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kp62\" (UniqueName: \"kubernetes.io/projected/a541c35c-1261-4779-bf69-775f8394b654-kube-api-access-4kp62\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835564 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835539    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-slash\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.835564 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835560    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-netns\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.835731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835585    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpthm\" (UniqueName: \"kubernetes.io/projected/5eced46d-ce11-4511-acd3-f8572157851d-kube-api-access-kpthm\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.835731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835623    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-host\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-tmp\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.835731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835677    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-host-slash\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.835731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835701    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-etc-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835769    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-config\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835784    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835793    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/485cb65a-3796-410d-a65a-19dc773993d2-agent-certs\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835819    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-modprobe-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835849    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysconfig\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835891    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-iptables-alerter-script\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835923    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-sys-fs\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835947    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-systemd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-ovn\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.835996    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836051    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-systemd\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836078    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-socket-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836101    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-device-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836123    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bcf9b29-dcc6-4882-bb12-2066011b34af-tmp-dir\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-kubelet\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836171    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5367657e-14dc-493e-a64a-5edc6117c72f-ovn-node-metrics-cert\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836192    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-run\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836213    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-sys\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836234    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-lib-modules\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836258    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dq2zz\" (UniqueName: \"kubernetes.io/projected/2da1e055-ff46-41d1-84a1-08432d943042-kube-api-access-dq2zz\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836315    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6bcf9b29-dcc6-4882-bb12-2066011b34af-hosts-file\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836341    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m2k4\" (UniqueName: \"kubernetes.io/projected/6bcf9b29-dcc6-4882-bb12-2066011b34af-kube-api-access-9m2k4\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836377    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-log-socket\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-script-lib\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836427    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-kubernetes\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.836601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836463    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-systemd-units\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836510    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-env-overrides\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836533    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-var-lib-kubelet\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836557    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqg5w\" (UniqueName: \"kubernetes.io/projected/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-kube-api-access-mqg5w\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836588    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-registration-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-etc-selinux\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836653    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-var-lib-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836685    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836707    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-node-log\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836745    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-bin\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836771    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-netd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836798    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.836925    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837129    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837144    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837375    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837440    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-4x8ff\""
Sep 11 13:04:47.837486 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837484    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837574    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837857    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-ntz2p\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.837989    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.838113    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-wv54k\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.838116    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-g6mrr\""
Sep 11 13:04:47.838437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.838127    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
Sep 11 13:04:47.838729 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.838655    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
Sep 11 13:04:47.840342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.839935    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:47.840342 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.840001    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:47.874783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.874691    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:47 +0000 UTC" deadline="2028-04-30 01:58:49.96247913 +0000 UTC"
Sep 11 13:04:47.874783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.874737    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14316h54m2.087760248s"
Sep 11 13:04:47.924698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.924669    2573 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world"
Sep 11 13:04:47.931546 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.931518    2573 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
Sep 11 13:04:47.937661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937630    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d664ec04-e3f3-46ae-a842-adb7cb6d7739-serviceca\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:47.937807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937680    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-slash\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.937807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937711    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpthm\" (UniqueName: \"kubernetes.io/projected/5eced46d-ce11-4511-acd3-f8572157851d-kube-api-access-kpthm\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.937807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937754    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dq2zz\" (UniqueName: \"kubernetes.io/projected/2da1e055-ff46-41d1-84a1-08432d943042-kube-api-access-dq2zz\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.937807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937783    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cnibin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.937807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-etc-kubernetes\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/485cb65a-3796-410d-a65a-19dc773993d2-agent-certs\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937829    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-slash\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937904    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysconfig\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937835    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysconfig\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937951    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-iptables-alerter-script\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.937981    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-conf-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938009    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-multus-certs\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938034    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-kubelet-config\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:47.938061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938062    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-systemd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938091    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938116    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-systemd\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938176    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-systemd\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938236    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-systemd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938330    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bcf9b29-dcc6-4882-bb12-2066011b34af-tmp-dir\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938360    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cni-binary-copy\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938378    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-daemon-config\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938402    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-kubelet\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-kubelet\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938510    2573 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938540    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5367657e-14dc-493e-a64a-5edc6117c72f-ovn-node-metrics-cert\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-run\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.938588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938589    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-sys\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938610    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-lib-modules\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938631    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m2k4\" (UniqueName: \"kubernetes.io/projected/6bcf9b29-dcc6-4882-bb12-2066011b34af-kube-api-access-9m2k4\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-kubernetes\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938654    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-run\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938668    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-iptables-alerter-script\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938807    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-lib-modules\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938881    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-sys\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938684    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-socket-dir-parent\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938934    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-bin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938964    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-systemd-units\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.938988    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-netns\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939016    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-node-log\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-etc-tuned\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939090    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tgq6r\" (UniqueName: \"kubernetes.io/projected/d664ec04-e3f3-46ae-a842-adb7cb6d7739-kube-api-access-tgq6r\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939117    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-kubernetes\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939121    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-dbus\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:47.939303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939166    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-systemd-units\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939168    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-klkm7\" (UniqueName: \"kubernetes.io/projected/5367657e-14dc-493e-a64a-5edc6117c72f-kube-api-access-klkm7\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939207    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/485cb65a-3796-410d-a65a-19dc773993d2-konnectivity-ca\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939234    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-system-cni-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939263    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-netns\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939288    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-host\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939327    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-tmp\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939351    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-host-slash\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939378    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-etc-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939391    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/6bcf9b29-dcc6-4882-bb12-2066011b34af-tmp-dir\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939413    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939503    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-node-log\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939738    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-run-netns\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939744    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-etc-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939801    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-host-slash\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939923    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-config\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939954    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-modprobe-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.940199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939982    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-sys-fs\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940007    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cnibin\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-multus\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940059    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5pp66\" (UniqueName: \"kubernetes.io/projected/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-kube-api-access-5pp66\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940084    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-ovn\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940109    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-socket-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940162    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-device-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940185    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-kubelet\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940206    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d664ec04-e3f3-46ae-a842-adb7cb6d7739-host\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940230    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6bcf9b29-dcc6-4882-bb12-2066011b34af-hosts-file\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940254    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940280    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-log-socket\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940305    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-script-lib\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940306    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-host\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.939844    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940329    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940372    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940403    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-system-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940429    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940457    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-env-overrides\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-socket-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940485    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-var-lib-kubelet\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mqg5w\" (UniqueName: \"kubernetes.io/projected/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-kube-api-access-mqg5w\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940534    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-device-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940560    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-registration-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940587    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-etc-selinux\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940622    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7b99\" (UniqueName: \"kubernetes.io/projected/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-kube-api-access-t7b99\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940635    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/485cb65a-3796-410d-a65a-19dc773993d2-konnectivity-ca\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940701    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-k8s-cni-cncf-io\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940712    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-ovn\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940724    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-kubelet-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940769    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-etc-selinux\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940727    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-hostroot\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:47.941906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940850    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-var-lib-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940908    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-bin\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940936    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-netd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.940994    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-bin\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941001    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-registration-dir\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941042    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-run-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941049    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/6bcf9b29-dcc6-4882-bb12-2066011b34af-hosts-file\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941081    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941141    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941149    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-env-overrides\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941156    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-host-cni-netd\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941168    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941222    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-var-lib-kubelet\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941233    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/2da1e055-ff46-41d1-84a1-08432d943042-sys-fs\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941237    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-modprobe-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.942434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941085    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-log-socket\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941109    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-config\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941267    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5367657e-14dc-493e-a64a-5edc6117c72f-var-lib-openvswitch\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941297    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-conf\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941324    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-d\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.941336    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941337    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4kp62\" (UniqueName: \"kubernetes.io/projected/a541c35c-1261-4779-bf69-775f8394b654-kube-api-access-4kp62\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941406    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/a541c35c-1261-4779-bf69-775f8394b654-etc-sysctl-conf\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.941453    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:48.441403074 +0000 UTC m=+2.961440421 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941480    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:47.943201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.941562    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5367657e-14dc-493e-a64a-5edc6117c72f-ovnkube-script-lib\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.943707 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.943333    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5367657e-14dc-493e-a64a-5edc6117c72f-ovn-node-metrics-cert\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.943801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.943760    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-etc-tuned\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.944090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.944069    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/485cb65a-3796-410d-a65a-19dc773993d2-agent-certs\") pod \"konnectivity-agent-j7r7w\" (UID: \"485cb65a-3796-410d-a65a-19dc773993d2\") " pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:47.945367 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.945348    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a541c35c-1261-4779-bf69-775f8394b654-tmp\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.950717 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.950687    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:47.950717 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.950712    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:47.950886 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.950725    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:47.950886 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:47.950802    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:48.450788032 +0000 UTC m=+2.970825377 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:47.953306 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.953284    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m2k4\" (UniqueName: \"kubernetes.io/projected/6bcf9b29-dcc6-4882-bb12-2066011b34af-kube-api-access-9m2k4\") pod \"node-resolver-zh8nr\" (UID: \"6bcf9b29-dcc6-4882-bb12-2066011b34af\") " pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:47.956387 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.956349    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kp62\" (UniqueName: \"kubernetes.io/projected/a541c35c-1261-4779-bf69-775f8394b654-kube-api-access-4kp62\") pod \"tuned-8fklw\" (UID: \"a541c35c-1261-4779-bf69-775f8394b654\") " pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:47.957251 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.957220    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpthm\" (UniqueName: \"kubernetes.io/projected/5eced46d-ce11-4511-acd3-f8572157851d-kube-api-access-kpthm\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:47.957559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.957535    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqg5w\" (UniqueName: \"kubernetes.io/projected/a9d7ad14-033f-4bf4-8d0e-03b832c138e3-kube-api-access-mqg5w\") pod \"iptables-alerter-wmvcd\" (UID: \"a9d7ad14-033f-4bf4-8d0e-03b832c138e3\") " pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:47.958521 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.958500    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dq2zz\" (UniqueName: \"kubernetes.io/projected/2da1e055-ff46-41d1-84a1-08432d943042-kube-api-access-dq2zz\") pod \"aws-ebs-csi-driver-node-nkq9z\" (UID: \"2da1e055-ff46-41d1-84a1-08432d943042\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:47.959568 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.959545    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-klkm7\" (UniqueName: \"kubernetes.io/projected/5367657e-14dc-493e-a64a-5edc6117c72f-kube-api-access-klkm7\") pod \"ovnkube-node-6lvhv\" (UID: \"5367657e-14dc-493e-a64a-5edc6117c72f\") " pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:47.970564 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.970518    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal" event={"ID":"af658a1e06ce3bdd72fe16298188a4e8","Type":"ContainerStarted","Data":"1c7df8816cf64cf53e307ccc7177d158895cb3062f5abf3546a7955d1d226953"}
Sep 11 13:04:47.972364 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:47.972344    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal" event={"ID":"9f5e10749336ffb5faa04f0c65e06cae","Type":"ContainerStarted","Data":"560a2dc3c3ea1bc615d1bf07cfbda92630364eab23f94ac32012f5f0df41c144"}
Sep 11 13:04:48.042118 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042082    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cnibin\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042124    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-multus\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042142    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5pp66\" (UniqueName: \"kubernetes.io/projected/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-kube-api-access-5pp66\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042182    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-kubelet\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042206    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d664ec04-e3f3-46ae-a842-adb7cb6d7739-host\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042210    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cnibin\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042232    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.042271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042260    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042283    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-system-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042304    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042331    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7b99\" (UniqueName: \"kubernetes.io/projected/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-kube-api-access-t7b99\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042354    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-k8s-cni-cncf-io\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042379    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-hostroot\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042416    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042439    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042465    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042489    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d664ec04-e3f3-46ae-a842-adb7cb6d7739-serviceca\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042490    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-multus\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042515    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cnibin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042547    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-kubelet\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042541    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-etc-kubernetes\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-conf-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042582    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d664ec04-e3f3-46ae-a842-adb7cb6d7739-host\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042589    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.042617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042593    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-etc-kubernetes\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042656    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-hostroot\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042717    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-conf-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.042809    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042861    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cnibin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.042873    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:48.542854392 +0000 UTC m=+3.062891725 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042913    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-multus-certs\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042944    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-kubelet-config\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042973    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cni-binary-copy\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.042999    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-daemon-config\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043029    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-socket-dir-parent\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043032    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-multus-certs\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043054    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-bin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043081    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-netns\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043107    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tgq6r\" (UniqueName: \"kubernetes.io/projected/d664ec04-e3f3-46ae-a842-adb7cb6d7739-kube-api-access-tgq6r\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-dbus\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043141    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/d664ec04-e3f3-46ae-a842-adb7cb6d7739-serviceca\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.043231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043160    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-system-cni-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043229    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-system-cni-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043270    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043356    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-dbus\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043403    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-cni-binary-copy\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043409    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-socket-dir-parent\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043273    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-netns\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043461    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-run-k8s-cni-cncf-io\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043466    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-host-var-lib-cni-bin\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043505    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b50d37b9-a041-4e48-97f6-65d919e4777d-kubelet-config\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043529    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-system-cni-dir\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043541    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043563    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-cni-binary-copy\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043545    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.044018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.043783    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-multus-daemon-config\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.058019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.057951    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tgq6r\" (UniqueName: \"kubernetes.io/projected/d664ec04-e3f3-46ae-a842-adb7cb6d7739-kube-api-access-tgq6r\") pod \"node-ca-nvkr7\" (UID: \"d664ec04-e3f3-46ae-a842-adb7cb6d7739\") " pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.058542 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.058517    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7b99\" (UniqueName: \"kubernetes.io/projected/d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2-kube-api-access-t7b99\") pod \"multus-additional-cni-plugins-xp7s6\" (UID: \"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2\") " pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.059011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.058987    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5pp66\" (UniqueName: \"kubernetes.io/projected/2412fb40-b2cf-4976-a3d4-5456fb7dc5c6-kube-api-access-5pp66\") pod \"multus-vxtcq\" (UID: \"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6\") " pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.130459 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.130424    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wmvcd"
Sep 11 13:04:48.137361 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.137333    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:04:48.151965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.151940    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:04:48.153088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.153068    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z"
Sep 11 13:04:48.160893 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.160876    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-zh8nr"
Sep 11 13:04:48.170905 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.170887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-8fklw"
Sep 11 13:04:48.177889 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.177867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xp7s6"
Sep 11 13:04:48.186794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.186769    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-nvkr7"
Sep 11 13:04:48.193589 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.193571    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-vxtcq"
Sep 11 13:04:48.445791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.445676    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:48.445938 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.445856    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:48.446034 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.445940    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:49.445915857 +0000 UTC m=+3.965953199 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:48.546067 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.546022    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:48.546067 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.546068    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:48.546323 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546162    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:48.546323 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546166    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:48.546323 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546235    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:49.546216356 +0000 UTC m=+4.066253701 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:48.546323 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546255    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:48.546323 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546273    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:48.546528 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.546333    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:49.546316883 +0000 UTC m=+4.066354215 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:48.876002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.875858    2573 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-09-10 12:59:47 +0000 UTC" deadline="2028-02-09 05:46:03.728276514 +0000 UTC"
Sep 11 13:04:48.876002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.875922    2573 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12376h41m14.852358811s"
Sep 11 13:04:48.966016 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:48.965988    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:48.966174 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:48.966100    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:49.054033 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.054007    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2412fb40_b2cf_4976_a3d4_5456fb7dc5c6.slice/crio-5616a6f11815d8995a641f19e0221a07a06eb2d2015e8c58b77c594005fc5954 WatchSource:0}: Error finding container 5616a6f11815d8995a641f19e0221a07a06eb2d2015e8c58b77c594005fc5954: Status 404 returned error can't find the container with id 5616a6f11815d8995a641f19e0221a07a06eb2d2015e8c58b77c594005fc5954
Sep 11 13:04:49.055643 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.055626    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2da1e055_ff46_41d1_84a1_08432d943042.slice/crio-1f4304505b789b52b05f275bbbff2aa3f7a152468cda776592e748be471b4d22 WatchSource:0}: Error finding container 1f4304505b789b52b05f275bbbff2aa3f7a152468cda776592e748be471b4d22: Status 404 returned error can't find the container with id 1f4304505b789b52b05f275bbbff2aa3f7a152468cda776592e748be471b4d22
Sep 11 13:04:49.057330 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.057298    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9d7ad14_033f_4bf4_8d0e_03b832c138e3.slice/crio-27d1adcb96cfd3acaa1be346bf54a2ad68016ce3101d78b8e20d57248ee6b0c6 WatchSource:0}: Error finding container 27d1adcb96cfd3acaa1be346bf54a2ad68016ce3101d78b8e20d57248ee6b0c6: Status 404 returned error can't find the container with id 27d1adcb96cfd3acaa1be346bf54a2ad68016ce3101d78b8e20d57248ee6b0c6
Sep 11 13:04:49.061160 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.061000    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd664ec04_e3f3_46ae_a842_adb7cb6d7739.slice/crio-0d4c35d002d6dc42fd7f46e0d89246146ba7ff81120dbf124f08698bee210e98 WatchSource:0}: Error finding container 0d4c35d002d6dc42fd7f46e0d89246146ba7ff81120dbf124f08698bee210e98: Status 404 returned error can't find the container with id 0d4c35d002d6dc42fd7f46e0d89246146ba7ff81120dbf124f08698bee210e98
Sep 11 13:04:49.063111 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.063033    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9906ab6_a3de_44bc_9be9_2ee9bbf7f2d2.slice/crio-4de94cf775c233ac619513368af61ada8f89c3c407ad6b2fe67d45e1d0b90d29 WatchSource:0}: Error finding container 4de94cf775c233ac619513368af61ada8f89c3c407ad6b2fe67d45e1d0b90d29: Status 404 returned error can't find the container with id 4de94cf775c233ac619513368af61ada8f89c3c407ad6b2fe67d45e1d0b90d29
Sep 11 13:04:49.064475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.064452    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod485cb65a_3796_410d_a65a_19dc773993d2.slice/crio-5212b8b94694efd63c1f12402aaa21fd524879fe61a72c63d1ddec120e8cf0ac WatchSource:0}: Error finding container 5212b8b94694efd63c1f12402aaa21fd524879fe61a72c63d1ddec120e8cf0ac: Status 404 returned error can't find the container with id 5212b8b94694efd63c1f12402aaa21fd524879fe61a72c63d1ddec120e8cf0ac
Sep 11 13:04:49.067020 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.065751    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5367657e_14dc_493e_a64a_5edc6117c72f.slice/crio-7060c23bed62bd387ac51a10f4f1548b9d27c6be393031fb04229b3e1fb1f155 WatchSource:0}: Error finding container 7060c23bed62bd387ac51a10f4f1548b9d27c6be393031fb04229b3e1fb1f155: Status 404 returned error can't find the container with id 7060c23bed62bd387ac51a10f4f1548b9d27c6be393031fb04229b3e1fb1f155
Sep 11 13:04:49.068416 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.068113    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6bcf9b29_dcc6_4882_bb12_2066011b34af.slice/crio-34559121fea83eb7480107d9755c7e88e092d329c1a70370ff97f9bc11284868 WatchSource:0}: Error finding container 34559121fea83eb7480107d9755c7e88e092d329c1a70370ff97f9bc11284868: Status 404 returned error can't find the container with id 34559121fea83eb7480107d9755c7e88e092d329c1a70370ff97f9bc11284868
Sep 11 13:04:49.068582 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:04:49.068556    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda541c35c_1261_4779_bf69_775f8394b654.slice/crio-e14f3dfe6a41eade5eacfaeaa7c808865f0de88c2888a9dda86a1b1fc5a242c1 WatchSource:0}: Error finding container e14f3dfe6a41eade5eacfaeaa7c808865f0de88c2888a9dda86a1b1fc5a242c1: Status 404 returned error can't find the container with id e14f3dfe6a41eade5eacfaeaa7c808865f0de88c2888a9dda86a1b1fc5a242c1
Sep 11 13:04:49.452658 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:49.452421    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:49.452824 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.452599    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:49.452824 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.452756    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:51.452739256 +0000 UTC m=+5.972776583 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:49.552956    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:49.553067    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553234    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553257    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553270    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553338    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:51.553317371 +0000 UTC m=+6.073354721 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:49.553362 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553357    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:49.553924 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.553401    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:51.553389484 +0000 UTC m=+6.073426813 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:49.967376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:49.966865    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:49.967376 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.967011    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:49.967376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:49.967101    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:49.967376 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:49.967216    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:50.001477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.000760    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal" event={"ID":"af658a1e06ce3bdd72fe16298188a4e8","Type":"ContainerStarted","Data":"47db8ce6426e1eb7ef794ba5be6e391ea2b49e20352d433d4fc515796199acc9"}
Sep 11 13:04:50.004700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.004399    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8fklw" event={"ID":"a541c35c-1261-4779-bf69-775f8394b654","Type":"ContainerStarted","Data":"e14f3dfe6a41eade5eacfaeaa7c808865f0de88c2888a9dda86a1b1fc5a242c1"}
Sep 11 13:04:50.017669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.017630    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zh8nr" event={"ID":"6bcf9b29-dcc6-4882-bb12-2066011b34af","Type":"ContainerStarted","Data":"34559121fea83eb7480107d9755c7e88e092d329c1a70370ff97f9bc11284868"}
Sep 11 13:04:50.021640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.021608    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerStarted","Data":"4de94cf775c233ac619513368af61ada8f89c3c407ad6b2fe67d45e1d0b90d29"}
Sep 11 13:04:50.033240 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.033208    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wmvcd" event={"ID":"a9d7ad14-033f-4bf4-8d0e-03b832c138e3","Type":"ContainerStarted","Data":"27d1adcb96cfd3acaa1be346bf54a2ad68016ce3101d78b8e20d57248ee6b0c6"}
Sep 11 13:04:50.038418 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.038298    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxtcq" event={"ID":"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6","Type":"ContainerStarted","Data":"5616a6f11815d8995a641f19e0221a07a06eb2d2015e8c58b77c594005fc5954"}
Sep 11 13:04:50.047583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.047035    2573 generic.go:356] "Generic (PLEG): container finished" podID="9f5e10749336ffb5faa04f0c65e06cae" containerID="79f8a8204c138ba14b478d751244d72995f41c03f3a6ab0c5f5cce43904f49fd" exitCode=0
Sep 11 13:04:50.047583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.047132    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal" event={"ID":"9f5e10749336ffb5faa04f0c65e06cae","Type":"ContainerDied","Data":"79f8a8204c138ba14b478d751244d72995f41c03f3a6ab0c5f5cce43904f49fd"}
Sep 11 13:04:50.057043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.057010    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"7060c23bed62bd387ac51a10f4f1548b9d27c6be393031fb04229b3e1fb1f155"}
Sep 11 13:04:50.064909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.064846    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-142-204.ec2.internal" podStartSLOduration=3.064827031 podStartE2EDuration="3.064827031s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:50.021464022 +0000 UTC m=+4.541501375" watchObservedRunningTime="2026-09-11 13:04:50.064827031 +0000 UTC m=+4.584864379"
Sep 11 13:04:50.072809 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.069286    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j7r7w" event={"ID":"485cb65a-3796-410d-a65a-19dc773993d2","Type":"ContainerStarted","Data":"5212b8b94694efd63c1f12402aaa21fd524879fe61a72c63d1ddec120e8cf0ac"}
Sep 11 13:04:50.086291 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.086254    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvkr7" event={"ID":"d664ec04-e3f3-46ae-a842-adb7cb6d7739","Type":"ContainerStarted","Data":"0d4c35d002d6dc42fd7f46e0d89246146ba7ff81120dbf124f08698bee210e98"}
Sep 11 13:04:50.107224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.107183    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z" event={"ID":"2da1e055-ff46-41d1-84a1-08432d943042","Type":"ContainerStarted","Data":"1f4304505b789b52b05f275bbbff2aa3f7a152468cda776592e748be471b4d22"}
Sep 11 13:04:50.966261 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:50.966226    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:50.966436 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:50.966368    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:51.125279 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.125239    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal" event={"ID":"9f5e10749336ffb5faa04f0c65e06cae","Type":"ContainerStarted","Data":"7d90a94fcc442079919d5092fecb85183088d50d263e97b99db29f42ce9f8ea4"}
Sep 11 13:04:51.159395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.159286    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-142-204.ec2.internal" podStartSLOduration=4.159260696 podStartE2EDuration="4.159260696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:04:51.159023943 +0000 UTC m=+5.679061292" watchObservedRunningTime="2026-09-11 13:04:51.159260696 +0000 UTC m=+5.679298034"
Sep 11 13:04:51.482466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.482375    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:51.482649 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.482525    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:51.482649 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.482592    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:55.482572358 +0000 UTC m=+10.002609682 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.582761    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.582820    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.582988    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.583063    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:04:55.583044116 +0000 UTC m=+10.103081445 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.583159    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.583175    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.583187    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:51.583598 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.583227    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:04:55.583213922 +0000 UTC m=+10.103251246 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:51.969080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.968243    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:51.969080 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.968382    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:51.969080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:51.968835    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:51.969080 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:51.968937    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:52.966258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:52.966222    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:52.966817 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:52.966382    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:53.967995 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:53.967958    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:53.968492 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:53.968077    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:53.968492 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:53.968365    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:53.968492 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:53.968439    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:54.965651 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:54.965560    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:54.965844 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:54.965744    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:55.519200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:55.519153    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:55.519705 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.519371    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:55.519705 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.519475    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:03.51942284 +0000 UTC m=+18.039460168 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:04:55.620356 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:55.620312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:55.620356 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:55.620356    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:55.620595 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620465    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:55.620595 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620522    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:03.620505412 +0000 UTC m=+18.140542749 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:04:55.620968 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620842    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:04:55.620968 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620869    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:04:55.620968 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620883    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:55.620968 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.620931    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:03.620915177 +0000 UTC m=+18.140952507 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:04:55.966906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:55.966821    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:55.967062 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.966939    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:55.967284 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:55.967149    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:55.967284 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:55.967248    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:56.966907 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:56.966357    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:56.966907 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:56.966508    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:57.965757 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:57.965720    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:57.965985 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:57.965854    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:57.965985 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:57.965911    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:57.966082 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:57.966008    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:04:58.965576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:58.965539    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:04:58.966059 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:58.965673    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:04:59.966531 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:59.966495    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:04:59.966978 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:59.966637    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:04:59.967041 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:04:59.966995    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:04:59.967113 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:04:59.967095    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:00.966457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:00.966424    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:00.966641 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:00.966560    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:01.966543 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:01.966511    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:01.966733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:01.966511    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:01.966733 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:01.966618    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:01.967019 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:01.966735    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:02.965922 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:02.965887    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:02.966142 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:02.966018    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:03.582332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:03.582299    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:03.582692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.582413    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:03.582692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.582468    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:19.582453918 +0000 UTC m=+34.102491241 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:03.682855 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:03.682822    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:03.682981 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:03.682863    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:03.682981 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.682973    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:05:03.683071 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.682994    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:05:03.683071 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.683003    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:03.683071 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.683008    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:03.683071 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.683051    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:19.683037879 +0000 UTC m=+34.203075202 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:03.683071 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.683065    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:19.683059314 +0000 UTC m=+34.203096636 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:03.966360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:03.966259    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:03.966360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:03.966304    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:03.966557 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.966398    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:03.966557 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:03.966516    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:04.965953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:04.965919    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:04.966346 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:04.966041    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:05.966992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:05.966670    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:05.967448 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:05.966749    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:05.967448 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:05.967072    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:05.967448 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:05.967174    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:06.966076 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:06.966030    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:06.966242 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:06.966173    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:07.966328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:07.966136    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:07.967052 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:07.966145    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:07.967052 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:07.966417    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:07.967052 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:07.966502    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:08.154804 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.154783    2573 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
Sep 11 13:05:08.176535 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.176505    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-nvkr7" event={"ID":"d664ec04-e3f3-46ae-a842-adb7cb6d7739","Type":"ContainerStarted","Data":"17c23c0f70f24330f0c51467a46f95280287d5e98d162c200d1738839d3f9d46"}
Sep 11 13:05:08.178437 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.178412    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z" event={"ID":"2da1e055-ff46-41d1-84a1-08432d943042","Type":"ContainerStarted","Data":"25e2c3b38e7eca22ce2acb19db8b866c3840f41a58d9c5642fde72d514c4552b"}
Sep 11 13:05:08.178536 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.178457    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z" event={"ID":"2da1e055-ff46-41d1-84a1-08432d943042","Type":"ContainerStarted","Data":"ab7bdf85f666cd539421671a92793730725f4c99fa009e4b0d3121f1b48ec16a"}
Sep 11 13:05:08.179862 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.179840    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-8fklw" event={"ID":"a541c35c-1261-4779-bf69-775f8394b654","Type":"ContainerStarted","Data":"69cb6b4ef0bc1b2007491b7b39e2b8c8aa71e61716acf472b2f5b7e41353cf27"}
Sep 11 13:05:08.181350 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.181328    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-zh8nr" event={"ID":"6bcf9b29-dcc6-4882-bb12-2066011b34af","Type":"ContainerStarted","Data":"ca34151596f9521992e4f693c81653b33b476b2aaa79373b9939331434ba3cce"}
Sep 11 13:05:08.182789 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.182769    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="d11f305769cac4a584d69abe5899b74ee80bccf15396d1a74cbad6f5e3f92c65" exitCode=0
Sep 11 13:05:08.182873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.182836    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"d11f305769cac4a584d69abe5899b74ee80bccf15396d1a74cbad6f5e3f92c65"}
Sep 11 13:05:08.184457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.184426    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-vxtcq" event={"ID":"2412fb40-b2cf-4976-a3d4-5456fb7dc5c6","Type":"ContainerStarted","Data":"3b41b57dd19568eebe839c773dea5a759798e77386673482cd514aa81e152e38"}
Sep 11 13:05:08.187495 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187475    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"a8bdabbb5300f2ffc6e44e595570368b0b8b07bfbbb217f98da446dbcad0bb70"}
Sep 11 13:05:08.187581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187535    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"60c82a7c7dc031df155647ae89e54e1a9e973d4c5da60dee91b5ca7c4293c4e2"}
Sep 11 13:05:08.187581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187577    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"2eae12991d3767ef955a1cb8b23687c728413e495403f452af7f1c67be9b096a"}
Sep 11 13:05:08.187667 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187618    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"a59c19b96dea23cf16139c4a57c60c9a9eb247939e988f1ca8048486b470cf88"}
Sep 11 13:05:08.187667 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187647    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"1449a8954e9a56e2dc155a586a3f22fa3fd58b3a49d0b1166d18c8c347e8a8c4"}
Sep 11 13:05:08.187765 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.187675    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"213f57fdfc1b42cebc06652ca66121fe18978d4534d867d4c3c3c4274b14393d"}
Sep 11 13:05:08.188807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.188786    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j7r7w" event={"ID":"485cb65a-3796-410d-a65a-19dc773993d2","Type":"ContainerStarted","Data":"408628683efcf5bba67de348101a8c203cba76d527184dd5193cd521c4f51d87"}
Sep 11 13:05:08.204924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.204877    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-nvkr7" podStartSLOduration=4.240465113 podStartE2EDuration="22.204864688s" totalImagesPullingTime="17.964399575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:08.204403283 +0000 UTC m=+22.724440628" watchObservedRunningTime="2026-09-11 13:05:08.204864688 +0000 UTC m=+22.724902032"
Sep 11 13:05:08.229587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.229539    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-zh8nr" podStartSLOduration=4.270493638 podStartE2EDuration="22.22952452s" totalImagesPullingTime="17.959030882s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:08.229189212 +0000 UTC m=+22.749226566" watchObservedRunningTime="2026-09-11 13:05:08.22952452 +0000 UTC m=+22.749561864"
Sep 11 13:05:08.287009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.286959    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-8fklw" podStartSLOduration=4.32969773 podStartE2EDuration="22.286943408s" totalImagesPullingTime="17.957245678s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:08.253648917 +0000 UTC m=+22.773686262" watchObservedRunningTime="2026-09-11 13:05:08.286943408 +0000 UTC m=+22.806980752"
Sep 11 13:05:08.287307 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.287288    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-vxtcq" podStartSLOduration=4.263898794 podStartE2EDuration="22.287284108s" totalImagesPullingTime="18.023385314s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:08.286557956 +0000 UTC m=+22.806595300" watchObservedRunningTime="2026-09-11 13:05:08.287284108 +0000 UTC m=+22.807321470"
Sep 11 13:05:08.310062 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.309970    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j7r7w" podStartSLOduration=4.348338442 podStartE2EDuration="22.309953938s" totalImagesPullingTime="17.961615496s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:08.309825164 +0000 UTC m=+22.829862508" watchObservedRunningTime="2026-09-11 13:05:08.309953938 +0000 UTC m=+22.829991265"
Sep 11 13:05:08.888899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.888771    2573 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-09-11T13:05:08.154799251Z","UUID":"04a339c6-ea5e-4f22-a8b8-51be06cbfe0c","Handler":null,"Name":"","Endpoint":""}
Sep 11 13:05:08.890675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.890640    2573 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
Sep 11 13:05:08.890675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.890676    2573 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
Sep 11 13:05:08.954698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.954670    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:05:08.955262 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.955241    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:05:08.966225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:08.966198    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:08.966352 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:08.966324    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:09.194530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.194435    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z" event={"ID":"2da1e055-ff46-41d1-84a1-08432d943042","Type":"ContainerStarted","Data":"0a128f6e2f1639b82062fcdc25d4ffeaec5ec993b6780272fc2058a9e555d5b7"}
Sep 11 13:05:09.196114 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.196079    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wmvcd" event={"ID":"a9d7ad14-033f-4bf4-8d0e-03b832c138e3","Type":"ContainerStarted","Data":"a261e1d3c778268199d2fba37be120ab9d215cc3925acb11da88ce2dc4f9cbc1"}
Sep 11 13:05:09.196941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.196915    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:05:09.197435 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.197414    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j7r7w"
Sep 11 13:05:09.262837 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.262782    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-nkq9z" podStartSLOduration=3.709804284 podStartE2EDuration="23.262765062s" totalImagesPullingTime="19.552960778s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:09.232466175 +0000 UTC m=+23.752503521" watchObservedRunningTime="2026-09-11 13:05:09.262765062 +0000 UTC m=+23.782802401"
Sep 11 13:05:09.263124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.263098    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wmvcd" podStartSLOduration=5.290257669 podStartE2EDuration="23.263092468s" totalImagesPullingTime="17.972834799s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:09.261647838 +0000 UTC m=+23.781685181" watchObservedRunningTime="2026-09-11 13:05:09.263092468 +0000 UTC m=+23.783129845"
Sep 11 13:05:09.966740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.966389    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:09.966740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:09.966442    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:09.967237 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:09.966756    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:09.967237 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:09.966829    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:10.202111 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:10.202072    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"453e35b60c00a3f6620b468d99a4703967a7d4d313cc22ba09e05460eeab6e19"}
Sep 11 13:05:10.965604 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:10.965572    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:10.965825 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:10.965725    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:11.965882 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:11.965838    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:11.965882 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:11.965878    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:11.966353 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:11.965975    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:11.966353 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:11.966083    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:12.212247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.211898    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" event={"ID":"5367657e-14dc-493e-a64a-5edc6117c72f","Type":"ContainerStarted","Data":"a9315411c28f7133ab697172ab407cddd42ac516225d46a7c8c09b87fcb72166"}
Sep 11 13:05:12.212748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.212371    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:12.212748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.212449    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:12.212748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.212501    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:12.232164 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.231953    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:12.232885 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.232867    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:12.252696 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.252557    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv" podStartSLOduration=8.215440912 podStartE2EDuration="26.252541144s" totalImagesPullingTime="18.037100232s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:12.248520958 +0000 UTC m=+26.768558304" watchObservedRunningTime="2026-09-11 13:05:12.252541144 +0000 UTC m=+26.772578489"
Sep 11 13:05:12.965626 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:12.965589    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:12.965794 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:12.965727    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:13.216311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:13.216224    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="984b77ff98d53c493297378ad602ca224466c0ab5c5552783fb6d51039015584" exitCode=0
Sep 11 13:05:13.216676 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:13.216313    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"984b77ff98d53c493297378ad602ca224466c0ab5c5552783fb6d51039015584"}
Sep 11 13:05:13.965904 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:13.965876    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:13.966030 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:13.966005    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:13.966095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:13.966060    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:13.966223 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:13.966200    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:14.221188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.221099    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="1a3b01094b473233928fd12109dc22993084c7a4557b31e1e336fd80ec58ea10" exitCode=0
Sep 11 13:05:14.221188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.221165    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"1a3b01094b473233928fd12109dc22993084c7a4557b31e1e336fd80ec58ea10"}
Sep 11 13:05:14.300960 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.300927    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dv8kv"]
Sep 11 13:05:14.301132 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.301046    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:14.301175 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:14.301142    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.305822    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jz9hm"]
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.305924    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v4ngl"]
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.306028    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:14.306199    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:14.306283    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:14.306512 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:14.306376    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:15.226528 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:15.226289    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="ff9cc48945ec5f46a9b993dbcafc62e178fe991ed13f5d518eb3dea05f79aeb2" exitCode=0
Sep 11 13:05:15.226528 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:15.226379    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"ff9cc48945ec5f46a9b993dbcafc62e178fe991ed13f5d518eb3dea05f79aeb2"}
Sep 11 13:05:15.966582 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:15.966549    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:15.966748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:15.966643    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:15.966810 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:15.966652    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:15.966810 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:15.966785    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:15.966909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:15.966832    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:15.966975 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:15.966904    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:17.965931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:17.965896    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:17.966389 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:17.965906    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:17.966389 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:17.966032    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:17.966389 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:17.966031    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-v4ngl" podUID="5eced46d-ce11-4511-acd3-f8572157851d"
Sep 11 13:05:17.966389 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:17.966114    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-dv8kv" podUID="b50d37b9-a041-4e48-97f6-65d919e4777d"
Sep 11 13:05:17.966389 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:17.966180    2573 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jz9hm" podUID="11362dd9-433c-4d61-9f6c-20b02f6b2345"
Sep 11 13:05:19.600653 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.600620    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:19.600993 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.600782    2573 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:19.600993 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.600849    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs podName:5eced46d-ce11-4511-acd3-f8572157851d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.600832779 +0000 UTC m=+66.120870107 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs") pod "network-metrics-daemon-v4ngl" (UID: "5eced46d-ce11-4511-acd3-f8572157851d") : object "openshift-multus"/"metrics-daemon-secret" not registered
Sep 11 13:05:19.701883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.701858    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:19.702014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.701890    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:19.702014 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.701979    2573 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:19.702124 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.702030    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret podName:b50d37b9-a041-4e48-97f6-65d919e4777d nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.702016379 +0000 UTC m=+66.222053702 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret") pod "global-pull-secret-syncer-dv8kv" (UID: "b50d37b9-a041-4e48-97f6-65d919e4777d") : object "kube-system"/"original-pull-secret" not registered
Sep 11 13:05:19.702124 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.702028    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
Sep 11 13:05:19.702124 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.702048    2573 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
Sep 11 13:05:19.702124 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.702057    2573 projected.go:196] Error preparing data for projected volume kube-api-access-xmcgd for pod openshift-network-diagnostics/network-check-target-jz9hm: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:19.702124 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:19.702096    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd podName:11362dd9-433c-4d61-9f6c-20b02f6b2345 nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.7020854 +0000 UTC m=+66.222122728 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xmcgd" (UniqueName: "kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd") pod "network-check-target-jz9hm" (UID: "11362dd9-433c-4d61-9f6c-20b02f6b2345") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
Sep 11 13:05:19.760756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.760731    2573 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-142-204.ec2.internal" event="NodeReady"
Sep 11 13:05:19.760893 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.760860    2573 kubelet_node_status.go:489] "Fast updating node status as it just became ready"
Sep 11 13:05:19.812407 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.812337    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-m4n6v"]
Sep 11 13:05:19.829281 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.829253    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-frj5r"]
Sep 11 13:05:19.829375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.829356    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:19.832543 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.832315    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
Sep 11 13:05:19.832543 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.832366    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
Sep 11 13:05:19.832757 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.832732    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-7xqq5\""
Sep 11 13:05:19.850454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.850421    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4n6v"]
Sep 11 13:05:19.850454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.850462    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-frj5r"]
Sep 11 13:05:19.850642 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.850539    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:19.853507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.853482    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-rl798\""
Sep 11 13:05:19.853507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.853495    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
Sep 11 13:05:19.853767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.853662    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
Sep 11 13:05:19.853767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.853726    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
Sep 11 13:05:19.858100 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.858078    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-87mdc"]
Sep 11 13:05:19.882526 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.882498    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-87mdc"]
Sep 11 13:05:19.882680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.882635    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:19.885764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.885738    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
Sep 11 13:05:19.885919 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.885775    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
Sep 11 13:05:19.886072 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.886056    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
Sep 11 13:05:19.886150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.886097    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
Sep 11 13:05:19.886150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.886123    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:19.949382 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.949352    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-fbscp"]
Sep 11 13:05:19.963003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.962978    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:19.966177 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.966151    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:19.966321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.966280    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:19.966421 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.966152    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:19.967758 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.967737    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
Sep 11 13:05:19.967881 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.967811    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
Sep 11 13:05:19.967942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.967917    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
Sep 11 13:05:19.968099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.968079    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
Sep 11 13:05:19.968287 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.968268    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:19.968631 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.968618    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-xtzgf\""
Sep 11 13:05:19.968899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.968878    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:19.968990 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.968886    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:05:19.969400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.969379    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:05:19.969495 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.969409    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:19.969674 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.969643    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:19.970199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:19.970184    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-fbscp"]
Sep 11 13:05:20.003452 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003418    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e28c135-924b-42c3-8197-930e3e65cd7e-tmp-dir\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.003452 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003454    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-stqqb\" (UniqueName: \"kubernetes.io/projected/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-api-access-stqqb\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.003621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003488    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22bpp\" (UniqueName: \"kubernetes.io/projected/2e28c135-924b-42c3-8197-930e3e65cd7e-kube-api-access-22bpp\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.003621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003507    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.003621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003576    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzdfk\" (UniqueName: \"kubernetes.io/projected/9e61091c-384a-40bd-a73f-7857eb641db4-kube-api-access-bzdfk\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.003621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003609    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.003762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003649    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e78d822-55ab-4ad7-906f-eb603b36b48a-crio-socket\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.003762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003667    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e78d822-55ab-4ad7-906f-eb603b36b48a-data-volume\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.003762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003686    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e61091c-384a-40bd-a73f-7857eb641db4-cert\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.003762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.003879 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.003767    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e28c135-924b-42c3-8197-930e3e65cd7e-config-volume\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.061565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.061530    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"]
Sep 11 13:05:20.081572 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.081492    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"]
Sep 11 13:05:20.081722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.081622    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"
Sep 11 13:05:20.084127 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.084110    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
Sep 11 13:05:20.084193 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.084176    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-d7mr8\""
Sep 11 13:05:20.084338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.084325    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
Sep 11 13:05:20.104789 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104754    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e78d822-55ab-4ad7-906f-eb603b36b48a-crio-socket\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.104789 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104787    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e78d822-55ab-4ad7-906f-eb603b36b48a-data-volume\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104828    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e61091c-384a-40bd-a73f-7857eb641db4-cert\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104902    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104945    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e28c135-924b-42c3-8197-930e3e65cd7e-config-volume\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104981    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pv5b8\" (UniqueName: \"kubernetes.io/projected/500f0a9a-2947-48b1-af8b-5c010c2f3860-kube-api-access-pv5b8\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.104997    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.104948    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/4e78d822-55ab-4ad7-906f-eb603b36b48a-crio-socket\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105013    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-cabundle\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.105106    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls podName:2e28c135-924b-42c3-8197-930e3e65cd7e nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.605081777 +0000 UTC m=+35.125119109 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls") pod "dns-default-m4n6v" (UID: "2e28c135-924b-42c3-8197-930e3e65cd7e") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e78d822-55ab-4ad7-906f-eb603b36b48a-data-volume\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105155    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e28c135-924b-42c3-8197-930e3e65cd7e-tmp-dir\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105178    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-stqqb\" (UniqueName: \"kubernetes.io/projected/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-api-access-stqqb\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105249    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22bpp\" (UniqueName: \"kubernetes.io/projected/2e28c135-924b-42c3-8197-930e3e65cd7e-kube-api-access-22bpp\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105274    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105306    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzdfk\" (UniqueName: \"kubernetes.io/projected/9e61091c-384a-40bd-a73f-7857eb641db4-kube-api-access-bzdfk\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105335    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105376    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-key\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.105380    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.105459    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.605446759 +0000 UTC m=+35.125484085 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.105467    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.105672 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.105531    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:20.605514348 +0000 UTC m=+35.125551674 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.105900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105686    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2e28c135-924b-42c3-8197-930e3e65cd7e-tmp-dir\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.105900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.105737    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/2e28c135-924b-42c3-8197-930e3e65cd7e-config-volume\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.109084 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.108965    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9e61091c-384a-40bd-a73f-7857eb641db4-cert\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.119953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.119924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-stqqb\" (UniqueName: \"kubernetes.io/projected/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-api-access-stqqb\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.120078 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.120010    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzdfk\" (UniqueName: \"kubernetes.io/projected/9e61091c-384a-40bd-a73f-7857eb641db4-kube-api-access-bzdfk\") pod \"ingress-canary-frj5r\" (UID: \"9e61091c-384a-40bd-a73f-7857eb641db4\") " pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.121476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.121453    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-22bpp\" (UniqueName: \"kubernetes.io/projected/2e28c135-924b-42c3-8197-930e3e65cd7e-kube-api-access-22bpp\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.160058 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.160024    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-frj5r"
Sep 11 13:05:20.205830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.205805    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-key\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.205924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.205865    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg9x7\" (UniqueName: \"kubernetes.io/projected/b3677a3a-ddc7-492a-9b5a-129e454031e8-kube-api-access-hg9x7\") pod \"migrator-8b4d697c5-hnfdh\" (UID: \"b3677a3a-ddc7-492a-9b5a-129e454031e8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"
Sep 11 13:05:20.205969 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.205920    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pv5b8\" (UniqueName: \"kubernetes.io/projected/500f0a9a-2947-48b1-af8b-5c010c2f3860-kube-api-access-pv5b8\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.206001 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.205981    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-cabundle\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.206588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.206567    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-cabundle\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.208404 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.208376    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/500f0a9a-2947-48b1-af8b-5c010c2f3860-signing-key\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.226516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.226480    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pv5b8\" (UniqueName: \"kubernetes.io/projected/500f0a9a-2947-48b1-af8b-5c010c2f3860-kube-api-access-pv5b8\") pod \"service-ca-568869fdc5-fbscp\" (UID: \"500f0a9a-2947-48b1-af8b-5c010c2f3860\") " pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.242338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.242305    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="259c7e85d89a77cfcb01904eb8d268b731a34c8b55a96988f233284511d55ad0" exitCode=0
Sep 11 13:05:20.242524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.242350    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"259c7e85d89a77cfcb01904eb8d268b731a34c8b55a96988f233284511d55ad0"}
Sep 11 13:05:20.279137 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.279108    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-568869fdc5-fbscp"
Sep 11 13:05:20.307441 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.306881    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg9x7\" (UniqueName: \"kubernetes.io/projected/b3677a3a-ddc7-492a-9b5a-129e454031e8-kube-api-access-hg9x7\") pod \"migrator-8b4d697c5-hnfdh\" (UID: \"b3677a3a-ddc7-492a-9b5a-129e454031e8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"
Sep 11 13:05:20.322196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.322168    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg9x7\" (UniqueName: \"kubernetes.io/projected/b3677a3a-ddc7-492a-9b5a-129e454031e8-kube-api-access-hg9x7\") pod \"migrator-8b4d697c5-hnfdh\" (UID: \"b3677a3a-ddc7-492a-9b5a-129e454031e8\") " pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"
Sep 11 13:05:20.389962 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.389928    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"
Sep 11 13:05:20.496101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.496068    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-frj5r"]
Sep 11 13:05:20.591559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.591470    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-568869fdc5-fbscp"]
Sep 11 13:05:20.596324 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:20.596296    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod500f0a9a_2947_48b1_af8b_5c010c2f3860.slice/crio-23a659566163570721878d2517fc92a73506683cd879e33306b64c9ccdcba683 WatchSource:0}: Error finding container 23a659566163570721878d2517fc92a73506683cd879e33306b64c9ccdcba683: Status 404 returned error can't find the container with id 23a659566163570721878d2517fc92a73506683cd879e33306b64c9ccdcba683
Sep 11 13:05:20.597501 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:20.597472    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e61091c_384a_40bd_a73f_7857eb641db4.slice/crio-f9bc4e1fcdc58c89c91e1dde7907ac6dd03f2d6386c7d4bee2f8aadd12b6823a WatchSource:0}: Error finding container f9bc4e1fcdc58c89c91e1dde7907ac6dd03f2d6386c7d4bee2f8aadd12b6823a: Status 404 returned error can't find the container with id f9bc4e1fcdc58c89c91e1dde7907ac6dd03f2d6386c7d4bee2f8aadd12b6823a
Sep 11 13:05:20.609055 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.609024    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.609080    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.609106    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609174    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609199    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609204    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609256    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.609240886 +0000 UTC m=+36.129278214 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609286    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.609266889 +0000 UTC m=+36.129304228 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:20.609698 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:20.609317    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls podName:2e28c135-924b-42c3-8197-930e3e65cd7e nodeName:}" failed. No retries permitted until 2026-09-11 13:05:21.609304763 +0000 UTC m=+36.129342087 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls") pod "dns-default-m4n6v" (UID: "2e28c135-924b-42c3-8197-930e3e65cd7e") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:20.867435 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:20.867188    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh"]
Sep 11 13:05:20.869965 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:20.869942    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb3677a3a_ddc7_492a_9b5a_129e454031e8.slice/crio-a69637fbd81bd7e8056b8cdae3cd81689ad95646a5ed13f2812946f32c5073d9 WatchSource:0}: Error finding container a69637fbd81bd7e8056b8cdae3cd81689ad95646a5ed13f2812946f32c5073d9: Status 404 returned error can't find the container with id a69637fbd81bd7e8056b8cdae3cd81689ad95646a5ed13f2812946f32c5073d9
Sep 11 13:05:21.130526 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.130444    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zh8nr_6bcf9b29-dcc6-4882-bb12-2066011b34af/dns-node-resolver/0.log"
Sep 11 13:05:21.247490 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.247449    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh" event={"ID":"b3677a3a-ddc7-492a-9b5a-129e454031e8","Type":"ContainerStarted","Data":"a69637fbd81bd7e8056b8cdae3cd81689ad95646a5ed13f2812946f32c5073d9"}
Sep 11 13:05:21.248886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.248853    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-fbscp" event={"ID":"500f0a9a-2947-48b1-af8b-5c010c2f3860","Type":"ContainerStarted","Data":"23a659566163570721878d2517fc92a73506683cd879e33306b64c9ccdcba683"}
Sep 11 13:05:21.250435 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.250407    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-frj5r" event={"ID":"9e61091c-384a-40bd-a73f-7857eb641db4","Type":"ContainerStarted","Data":"f9bc4e1fcdc58c89c91e1dde7907ac6dd03f2d6386c7d4bee2f8aadd12b6823a"}
Sep 11 13:05:21.254913 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.254843    2573 generic.go:356] "Generic (PLEG): container finished" podID="d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2" containerID="536790ec8f22c82fec6a3a670e6cd852f49ee72f8db1699daab9d6d0cb7ce1f4" exitCode=0
Sep 11 13:05:21.255050 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.254920    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerDied","Data":"536790ec8f22c82fec6a3a670e6cd852f49ee72f8db1699daab9d6d0cb7ce1f4"}
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.618140    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618213    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.618244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618278    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls podName:2e28c135-924b-42c3-8197-930e3e65cd7e nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.618259025 +0000 UTC m=+38.138296365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls") pod "dns-default-m4n6v" (UID: "2e28c135-924b-42c3-8197-930e3e65cd7e") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618306    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.618316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618366    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.618354533 +0000 UTC m=+38.138391856 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:21.618417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618408    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.619231 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:21.618439    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:23.61842956 +0000 UTC m=+38.138466884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:21.703794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:21.703585    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvkr7_d664ec04-e3f3-46ae-a842-adb7cb6d7739/node-ca/0.log"
Sep 11 13:05:22.260211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:22.260181    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" event={"ID":"d9906ab6-a3de-44bc-9be9-2ee9bbf7f2d2","Type":"ContainerStarted","Data":"bd9137330b064dec95901ceb507f05c58e612b2be0a1247abcb444617df3eec3"}
Sep 11 13:05:22.284930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:22.284869    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xp7s6" podStartSLOduration=8.615982495 podStartE2EDuration="36.284853625s" totalImagesPullingTime="27.66887113s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:22.28357283 +0000 UTC m=+36.803610174" watchObservedRunningTime="2026-09-11 13:05:22.284853625 +0000 UTC m=+36.804890970"
Sep 11 13:05:23.634952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:23.634917    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:23.634979    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:23.635017    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635103    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635128    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635111    2573 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635185    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.635163108 +0000 UTC m=+42.155200446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635207    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls podName:2e28c135-924b-42c3-8197-930e3e65cd7e nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.635198116 +0000 UTC m=+42.155235438 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls") pod "dns-default-m4n6v" (UID: "2e28c135-924b-42c3-8197-930e3e65cd7e") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:23.635403 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:23.635222    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:27.635213577 +0000 UTC m=+42.155250903 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : configmap "kube-rbac-proxy" not found
Sep 11 13:05:24.267684 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.267480    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-frj5r" event={"ID":"9e61091c-384a-40bd-a73f-7857eb641db4","Type":"ContainerStarted","Data":"4a739de92a6a112325a3fcabfbf077337c47369f6cd05a4277bbfd8e3922d328"}
Sep 11 13:05:24.269378 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.269353    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh" event={"ID":"b3677a3a-ddc7-492a-9b5a-129e454031e8","Type":"ContainerStarted","Data":"e6030d710f6066b276c2a3d8a461e61bbf837b8fb077f815bded81958e7a5c3c"}
Sep 11 13:05:24.269494 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.269404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh" event={"ID":"b3677a3a-ddc7-492a-9b5a-129e454031e8","Type":"ContainerStarted","Data":"d2cf0abf98968a4b1acc627745347a5a7d752a884c453580bb1817cb7fe64d36"}
Sep 11 13:05:24.270876 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.270853    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-568869fdc5-fbscp" event={"ID":"500f0a9a-2947-48b1-af8b-5c010c2f3860","Type":"ContainerStarted","Data":"f4af2b58fefa0e1ce6d57c918fde8f007980987cdeb765410b74fbe339df1b5f"}
Sep 11 13:05:24.294381 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.294327    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-frj5r" podStartSLOduration=2.058562293 podStartE2EDuration="5.294312209s" totalImagesPullingTime="3.235749916s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:24.293977142 +0000 UTC m=+38.814014488" watchObservedRunningTime="2026-09-11 13:05:24.294312209 +0000 UTC m=+38.814349553"
Sep 11 13:05:24.317003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.316955    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-568869fdc5-fbscp" podStartSLOduration=2.074630232 podStartE2EDuration="5.316939767s" totalImagesPullingTime="3.242309535s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:24.316632134 +0000 UTC m=+38.836669480" watchObservedRunningTime="2026-09-11 13:05:24.316939767 +0000 UTC m=+38.836977111"
Sep 11 13:05:24.334788 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:24.334740    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-8b4d697c5-hnfdh" podStartSLOduration=1.371881283 podStartE2EDuration="4.334724714s" totalImagesPullingTime="2.962843431s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:24.333981562 +0000 UTC m=+38.854018898" watchObservedRunningTime="2026-09-11 13:05:24.334724714 +0000 UTC m=+38.854762068"
Sep 11 13:05:27.666093 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:27.666053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:27.666122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:27.666163    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:27.666208    2573 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:27.666252    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:27.666279    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls podName:2e28c135-924b-42c3-8197-930e3e65cd7e nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.666261584 +0000 UTC m=+50.186298927 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls") pod "dns-default-m4n6v" (UID: "2e28c135-924b-42c3-8197-930e3e65cd7e") : secret "dns-default-metrics-tls" not found
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:27.666296    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:35.666288041 +0000 UTC m=+50.186325366 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:27.666692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:27.666665    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/4e78d822-55ab-4ad7-906f-eb603b36b48a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:35.723285 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:35.723248    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:35.723728 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:35.723334    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:35.723728 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:35.723471    2573 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.723728 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:05:35.723539    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls podName:4e78d822-55ab-4ad7-906f-eb603b36b48a nodeName:}" failed. No retries permitted until 2026-09-11 13:05:51.723520718 +0000 UTC m=+66.243558044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls") pod "insights-runtime-extractor-87mdc" (UID: "4e78d822-55ab-4ad7-906f-eb603b36b48a") : secret "insights-runtime-extractor-tls" not found
Sep 11 13:05:35.727290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:35.727264    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/2e28c135-924b-42c3-8197-930e3e65cd7e-metrics-tls\") pod \"dns-default-m4n6v\" (UID: \"2e28c135-924b-42c3-8197-930e3e65cd7e\") " pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:35.740065 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:35.740043    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:36.033298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:36.033267    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-m4n6v"]
Sep 11 13:05:36.037017 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:36.036994    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2e28c135_924b_42c3_8197_930e3e65cd7e.slice/crio-76e90ad701f5425025d288fc73d7a810b195e02e5354f51cd1fd1b98caa616a9 WatchSource:0}: Error finding container 76e90ad701f5425025d288fc73d7a810b195e02e5354f51cd1fd1b98caa616a9: Status 404 returned error can't find the container with id 76e90ad701f5425025d288fc73d7a810b195e02e5354f51cd1fd1b98caa616a9
Sep 11 13:05:36.302666 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:36.302583    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4n6v" event={"ID":"2e28c135-924b-42c3-8197-930e3e65cd7e","Type":"ContainerStarted","Data":"76e90ad701f5425025d288fc73d7a810b195e02e5354f51cd1fd1b98caa616a9"}
Sep 11 13:05:38.311023 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:38.310926    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4n6v" event={"ID":"2e28c135-924b-42c3-8197-930e3e65cd7e","Type":"ContainerStarted","Data":"1922b8768a2111a3904f653d4bcbd77867982fba91971736c61faab9d0288fd2"}
Sep 11 13:05:38.311023 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:38.310981    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-m4n6v" event={"ID":"2e28c135-924b-42c3-8197-930e3e65cd7e","Type":"ContainerStarted","Data":"b719ac7e6958a6f6c3dd9a0a3e3d22ca771a72e138dbb8dc4564dc1f64626744"}
Sep 11 13:05:38.311457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:38.311057    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:41.183891 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.183827    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-m4n6v" podStartSLOduration=20.22599375 podStartE2EDuration="22.183803113s" totalImagesPullingTime="1.957809363s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:38.333994832 +0000 UTC m=+52.854032177" watchObservedRunningTime="2026-09-11 13:05:41.183803113 +0000 UTC m=+55.703840457"
Sep 11 13:05:41.184466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.184434    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"]
Sep 11 13:05:41.186626 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.186610    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"]
Sep 11 13:05:41.186810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.186792    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.188743 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.188700    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.193753 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.193723    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
Sep 11 13:05:41.193890 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.193759    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-wz4xx\""
Sep 11 13:05:41.194423 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.194407    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\""
Sep 11 13:05:41.194539 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.194521    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
Sep 11 13:05:41.194614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.194600    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\""
Sep 11 13:05:41.194906 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.194892    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
Sep 11 13:05:41.195140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.195122    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\""
Sep 11 13:05:41.197896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.197878    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
Sep 11 13:05:41.201953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.201932    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"]
Sep 11 13:05:41.203868 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.203851    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\""
Sep 11 13:05:41.210811 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.210792    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"]
Sep 11 13:05:41.262994 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.262964    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263002    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263025    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263046    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263086    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tc4kx\" (UniqueName: \"kubernetes.io/projected/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-kube-api-access-tc4kx\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.263159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263135    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bgvs\" (UniqueName: \"kubernetes.io/projected/c379c1b2-d2a0-4928-a117-3af733a5c57f-kube-api-access-2bgvs\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.263347 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.263197    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c379c1b2-d2a0-4928-a117-3af733a5c57f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.364363 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364328    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tc4kx\" (UniqueName: \"kubernetes.io/projected/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-kube-api-access-tc4kx\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.364363 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364371    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bgvs\" (UniqueName: \"kubernetes.io/projected/c379c1b2-d2a0-4928-a117-3af733a5c57f-kube-api-access-2bgvs\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364425    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c379c1b2-d2a0-4928-a117-3af733a5c57f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364477    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364519    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364538    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.364629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.364563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.365509 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.365483    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-additional-service-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.367252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.367224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.367491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.367467    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-ca\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.367566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.367551    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.367638 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.367618    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-hub\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.367670 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.367631    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c379c1b2-d2a0-4928-a117-3af733a5c57f-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.374235 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.374213    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tc4kx\" (UniqueName: \"kubernetes.io/projected/c4051c7d-2ca4-4e98-bb3e-6c4e24849efe-kube-api-access-tc4kx\") pod \"cluster-proxy-proxy-agent-59fdf74cf4-69w2w\" (UID: \"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.375400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.375381    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bgvs\" (UniqueName: \"kubernetes.io/projected/c379c1b2-d2a0-4928-a117-3af733a5c57f-kube-api-access-2bgvs\") pod \"managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc\" (UID: \"c379c1b2-d2a0-4928-a117-3af733a5c57f\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.507490 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.507460    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"
Sep 11 13:05:41.513139 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.513114    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"
Sep 11 13:05:41.786530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.786449    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc"]
Sep 11 13:05:41.789925 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:41.789894    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc379c1b2_d2a0_4928_a117_3af733a5c57f.slice/crio-ecaf25dc4ac85c61dc8fa24b50f8ab55e118f7b7aa2db40ef412a3cad29476a7 WatchSource:0}: Error finding container ecaf25dc4ac85c61dc8fa24b50f8ab55e118f7b7aa2db40ef412a3cad29476a7: Status 404 returned error can't find the container with id ecaf25dc4ac85c61dc8fa24b50f8ab55e118f7b7aa2db40ef412a3cad29476a7
Sep 11 13:05:41.808332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:41.808298    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w"]
Sep 11 13:05:41.812531 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:41.812499    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc4051c7d_2ca4_4e98_bb3e_6c4e24849efe.slice/crio-2d7802fa466e038a8767bbb3c1ef29998163fef5783cb944c2115d6c39db4eb4 WatchSource:0}: Error finding container 2d7802fa466e038a8767bbb3c1ef29998163fef5783cb944c2115d6c39db4eb4: Status 404 returned error can't find the container with id 2d7802fa466e038a8767bbb3c1ef29998163fef5783cb944c2115d6c39db4eb4
Sep 11 13:05:42.325453 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:42.325404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc" event={"ID":"c379c1b2-d2a0-4928-a117-3af733a5c57f","Type":"ContainerStarted","Data":"ecaf25dc4ac85c61dc8fa24b50f8ab55e118f7b7aa2db40ef412a3cad29476a7"}
Sep 11 13:05:42.326519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:42.326482    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w" event={"ID":"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe","Type":"ContainerStarted","Data":"2d7802fa466e038a8767bbb3c1ef29998163fef5783cb944c2115d6c39db4eb4"}
Sep 11 13:05:44.233720 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:44.233672    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-6lvhv"
Sep 11 13:05:46.342622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.342578    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc" event={"ID":"c379c1b2-d2a0-4928-a117-3af733a5c57f","Type":"ContainerStarted","Data":"a2506e7211282800c82048ae13defba5f78b3acd0c6426c5c01347bdad9537ac"}
Sep 11 13:05:46.344899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.344872    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w" event={"ID":"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe","Type":"ContainerStarted","Data":"9eb23ff8508c87b4a716da82c1d47a50bff1d572d479199377a91bed471de687"}
Sep 11 13:05:46.345053 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.344912    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w" event={"ID":"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe","Type":"ContainerStarted","Data":"5d8cc7d1d0f620b1d147e4db21f1aff376ca066beb8640614049d007ae990d64"}
Sep 11 13:05:46.345053 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.344934    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w" event={"ID":"c4051c7d-2ca4-4e98-bb3e-6c4e24849efe","Type":"ContainerStarted","Data":"13cc32e30484b4fcee5ee9c225a4b46fc8f7568d98ffad942d504df010fcb09c"}
Sep 11 13:05:46.363658 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.363613    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-6df6fb46cd-4jhgc" podStartSLOduration=1.353806456 podStartE2EDuration="5.363599899s" totalImagesPullingTime="4.009793443s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:46.363049723 +0000 UTC m=+60.883087068" watchObservedRunningTime="2026-09-11 13:05:46.363599899 +0000 UTC m=+60.883637244"
Sep 11 13:05:46.384224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:46.384178    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-59fdf74cf4-69w2w" podStartSLOduration=1.3855742819999999 podStartE2EDuration="5.384163676s" totalImagesPullingTime="3.998589394s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:05:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:46.383504237 +0000 UTC m=+60.903541582" watchObservedRunningTime="2026-09-11 13:05:46.384163676 +0000 UTC m=+60.904201021"
Sep 11 13:05:48.316637 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:48.316608    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-m4n6v"
Sep 11 13:05:51.645691 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.645657    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:51.648124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.648106    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
Sep 11 13:05:51.658131 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.658110    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/5eced46d-ce11-4511-acd3-f8572157851d-metrics-certs\") pod \"network-metrics-daemon-v4ngl\" (UID: \"5eced46d-ce11-4511-acd3-f8572157851d\") " pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:51.746258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.746230    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:51.746363 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.746261    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:51.746363 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.746281    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:51.748428 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.748404    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/4e78d822-55ab-4ad7-906f-eb603b36b48a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-87mdc\" (UID: \"4e78d822-55ab-4ad7-906f-eb603b36b48a\") " pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:51.749105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.749090    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
Sep 11 13:05:51.749181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.749095    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
Sep 11 13:05:51.758568 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.758545    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b50d37b9-a041-4e48-97f6-65d919e4777d-original-pull-secret\") pod \"global-pull-secret-syncer-dv8kv\" (UID: \"b50d37b9-a041-4e48-97f6-65d919e4777d\") " pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:51.759077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.759061    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
Sep 11 13:05:51.768671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.768651    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmcgd\" (UniqueName: \"kubernetes.io/projected/11362dd9-433c-4d61-9f6c-20b02f6b2345-kube-api-access-xmcgd\") pod \"network-check-target-jz9hm\" (UID: \"11362dd9-433c-4d61-9f6c-20b02f6b2345\") " pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:51.787674 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.787654    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-lg7nk\""
Sep 11 13:05:51.795388 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.795367    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-np5mv\""
Sep 11 13:05:51.795483 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.795406    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-dv8kv"
Sep 11 13:05:51.796066 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.796048    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-v4ngl"
Sep 11 13:05:51.802036 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.802019    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:51.993525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:51.993501    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-jddd5\""
Sep 11 13:05:52.002300 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.002281    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-87mdc"
Sep 11 13:05:52.078873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.078766    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jz9hm"]
Sep 11 13:05:52.081608 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:52.081582    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod11362dd9_433c_4d61_9f6c_20b02f6b2345.slice/crio-23ee64f46f88609ddee30563991423e6dcb8e9b4e0cd2af65ae74340e61db173 WatchSource:0}: Error finding container 23ee64f46f88609ddee30563991423e6dcb8e9b4e0cd2af65ae74340e61db173: Status 404 returned error can't find the container with id 23ee64f46f88609ddee30563991423e6dcb8e9b4e0cd2af65ae74340e61db173
Sep 11 13:05:52.082340 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.082314    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-dv8kv"]
Sep 11 13:05:52.083361 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.083340    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-v4ngl"]
Sep 11 13:05:52.084805 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:52.084778    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5eced46d_ce11_4511_acd3_f8572157851d.slice/crio-529c597013ba4b6d97dcbed14d5cedddda0bc37cd377c5a3bdb606a682969f90 WatchSource:0}: Error finding container 529c597013ba4b6d97dcbed14d5cedddda0bc37cd377c5a3bdb606a682969f90: Status 404 returned error can't find the container with id 529c597013ba4b6d97dcbed14d5cedddda0bc37cd377c5a3bdb606a682969f90
Sep 11 13:05:52.085475 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:52.085458    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb50d37b9_a041_4e48_97f6_65d919e4777d.slice/crio-ef02e57b4c20aa148c2ca3d04883c460683ec8f6632c8be2b97dba014692b840 WatchSource:0}: Error finding container ef02e57b4c20aa148c2ca3d04883c460683ec8f6632c8be2b97dba014692b840: Status 404 returned error can't find the container with id ef02e57b4c20aa148c2ca3d04883c460683ec8f6632c8be2b97dba014692b840
Sep 11 13:05:52.281117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.281052    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-87mdc"]
Sep 11 13:05:52.285661 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:05:52.285643    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e78d822_55ab_4ad7_906f_eb603b36b48a.slice/crio-994cf08ff7a7823345aaa91822ff8bef4af4c2837ad9f66c62d24a958df70aed WatchSource:0}: Error finding container 994cf08ff7a7823345aaa91822ff8bef4af4c2837ad9f66c62d24a958df70aed: Status 404 returned error can't find the container with id 994cf08ff7a7823345aaa91822ff8bef4af4c2837ad9f66c62d24a958df70aed
Sep 11 13:05:52.365315 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.365284    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jz9hm" event={"ID":"11362dd9-433c-4d61-9f6c-20b02f6b2345","Type":"ContainerStarted","Data":"23ee64f46f88609ddee30563991423e6dcb8e9b4e0cd2af65ae74340e61db173"}
Sep 11 13:05:52.366942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.366920    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-87mdc" event={"ID":"4e78d822-55ab-4ad7-906f-eb603b36b48a","Type":"ContainerStarted","Data":"abfd9eb6f8bb13804c3b471c99894d7345bcb9ec67a5925c484a452dc6db8354"}
Sep 11 13:05:52.367034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.366970    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-87mdc" event={"ID":"4e78d822-55ab-4ad7-906f-eb603b36b48a","Type":"ContainerStarted","Data":"994cf08ff7a7823345aaa91822ff8bef4af4c2837ad9f66c62d24a958df70aed"}
Sep 11 13:05:52.368123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.368100    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dv8kv" event={"ID":"b50d37b9-a041-4e48-97f6-65d919e4777d","Type":"ContainerStarted","Data":"ef02e57b4c20aa148c2ca3d04883c460683ec8f6632c8be2b97dba014692b840"}
Sep 11 13:05:52.369163 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:52.369143    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4ngl" event={"ID":"5eced46d-ce11-4511-acd3-f8572157851d","Type":"ContainerStarted","Data":"529c597013ba4b6d97dcbed14d5cedddda0bc37cd377c5a3bdb606a682969f90"}
Sep 11 13:05:53.375617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:53.375583    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4ngl" event={"ID":"5eced46d-ce11-4511-acd3-f8572157851d","Type":"ContainerStarted","Data":"48560a808fa6b1a46eb24a386e0be8038c94f385ebd8c5fc044369ec9da28312"}
Sep 11 13:05:53.378161 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:53.378136    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-87mdc" event={"ID":"4e78d822-55ab-4ad7-906f-eb603b36b48a","Type":"ContainerStarted","Data":"e04a62311bd45df70f96c196fc242542339157294ae5c9c8605529a5e8a2a08e"}
Sep 11 13:05:54.386334 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:54.386241    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-v4ngl" event={"ID":"5eced46d-ce11-4511-acd3-f8572157851d","Type":"ContainerStarted","Data":"7de6bd8123567a4844f4d46a9b0cc7110d3e14dce249b57323559806b372f379"}
Sep 11 13:05:54.411034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:54.410984    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-v4ngl" podStartSLOduration=67.316687169 podStartE2EDuration="1m8.410966236s" totalImagesPullingTime="1.094279067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:54.40841713 +0000 UTC m=+68.928454478" watchObservedRunningTime="2026-09-11 13:05:54.410966236 +0000 UTC m=+68.931003582"
Sep 11 13:05:58.401315 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.401270    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-87mdc" event={"ID":"4e78d822-55ab-4ad7-906f-eb603b36b48a","Type":"ContainerStarted","Data":"995baa12dea591cc6d0ac91ed2b8bd371b050264002c2ba19059cc38ab65f02b"}
Sep 11 13:05:58.402927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.402899    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-dv8kv" event={"ID":"b50d37b9-a041-4e48-97f6-65d919e4777d","Type":"ContainerStarted","Data":"9f8b9d580971ee2208064540de23c95141120227fb56a9a8fe1402c4fd07bed5"}
Sep 11 13:05:58.404356 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.404331    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jz9hm" event={"ID":"11362dd9-433c-4d61-9f6c-20b02f6b2345","Type":"ContainerStarted","Data":"c6caf46dddc2caf6c8f87da9f9a8d8e876b9a912ed4082d11a5c4de30b6bec95"}
Sep 11 13:05:58.404472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.404460    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:05:58.425562 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.425523    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-87mdc" podStartSLOduration=34.465752781 podStartE2EDuration="39.425507824s" totalImagesPullingTime="4.959755043s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:05:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:58.424933341 +0000 UTC m=+72.944970685" watchObservedRunningTime="2026-09-11 13:05:58.425507824 +0000 UTC m=+72.945545173"
Sep 11 13:05:58.445141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.445107    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-dv8kv" podStartSLOduration=67.139146819 podStartE2EDuration="1m12.445095925s" totalImagesPullingTime="5.305949106s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:58.444291162 +0000 UTC m=+72.964328519" watchObservedRunningTime="2026-09-11 13:05:58.445095925 +0000 UTC m=+72.965133269"
Sep 11 13:05:58.465122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:05:58.465093    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jz9hm" podStartSLOduration=67.168091724 podStartE2EDuration="1m12.4650814s" totalImagesPullingTime="5.296989676s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:04:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:05:58.464942832 +0000 UTC m=+72.984980177" watchObservedRunningTime="2026-09-11 13:05:58.4650814 +0000 UTC m=+72.985118723"
Sep 11 13:06:06.597432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.597395    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-ksck6"]
Sep 11 13:06:06.600411 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.600389    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.604349 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.604328    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
Sep 11 13:06:06.604510 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.604487    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
Sep 11 13:06:06.606111 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.606092    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
Sep 11 13:06:06.606341 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.606312    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
Sep 11 13:06:06.607970 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.607950    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6lgh2\""
Sep 11 13:06:06.608546 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.608523    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
Sep 11 13:06:06.612956 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.612941    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
Sep 11 13:06:06.650972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.650935    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-wtmp\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.650972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.650979    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-root\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651042    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651096    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-metrics-client-ca\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651123    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tsr4v\" (UniqueName: \"kubernetes.io/projected/a5e50d9f-ce30-4137-9af0-1982af45e24e-kube-api-access-tsr4v\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651152    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651226    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-textfile\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651272    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-tls\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.651325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.651301    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-sys\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752391 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752346    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-metrics-client-ca\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752391 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752397    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tsr4v\" (UniqueName: \"kubernetes.io/projected/a5e50d9f-ce30-4137-9af0-1982af45e24e-kube-api-access-tsr4v\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752575    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752624    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-textfile\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752689    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-tls\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752740    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-sys\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752775    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-wtmp\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752795    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-root\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752815    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.752941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752902    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-sys\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.753089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-root\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.753089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752968    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-wtmp\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.753089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.752973    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-textfile\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.753302 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.753285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-metrics-client-ca\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.753383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.753333    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-accelerators-collector-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.754968 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.754947    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.755195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.755174    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a5e50d9f-ce30-4137-9af0-1982af45e24e-node-exporter-tls\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.767852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.767827    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tsr4v\" (UniqueName: \"kubernetes.io/projected/a5e50d9f-ce30-4137-9af0-1982af45e24e-kube-api-access-tsr4v\") pod \"node-exporter-ksck6\" (UID: \"a5e50d9f-ce30-4137-9af0-1982af45e24e\") " pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.909770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:06.909670    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-ksck6"
Sep 11 13:06:06.921509 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:06:06.921479    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5e50d9f_ce30_4137_9af0_1982af45e24e.slice/crio-cb470180ffd2247dad0c388008b4fc92ed67ae163dc558cdd0a994dc1982f2cc WatchSource:0}: Error finding container cb470180ffd2247dad0c388008b4fc92ed67ae163dc558cdd0a994dc1982f2cc: Status 404 returned error can't find the container with id cb470180ffd2247dad0c388008b4fc92ed67ae163dc558cdd0a994dc1982f2cc
Sep 11 13:06:07.443163 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:07.443120    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksck6" event={"ID":"a5e50d9f-ce30-4137-9af0-1982af45e24e","Type":"ContainerStarted","Data":"cb470180ffd2247dad0c388008b4fc92ed67ae163dc558cdd0a994dc1982f2cc"}
Sep 11 13:06:08.448588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:08.448552    2573 generic.go:356] "Generic (PLEG): container finished" podID="a5e50d9f-ce30-4137-9af0-1982af45e24e" containerID="e565f69cf788f5de2b83dc42e3cf52f11b9acc0711317ff93939ace58249692b" exitCode=0
Sep 11 13:06:08.449035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:08.448646    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksck6" event={"ID":"a5e50d9f-ce30-4137-9af0-1982af45e24e","Type":"ContainerDied","Data":"e565f69cf788f5de2b83dc42e3cf52f11b9acc0711317ff93939ace58249692b"}
Sep 11 13:06:09.454538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:09.454505    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksck6" event={"ID":"a5e50d9f-ce30-4137-9af0-1982af45e24e","Type":"ContainerStarted","Data":"1caaac4f2ffdf9c2e3aac2b5dca0e9f9c3d359009593ea8b090278f004e9fb60"}
Sep 11 13:06:09.454934 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:09.454556    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-ksck6" event={"ID":"a5e50d9f-ce30-4137-9af0-1982af45e24e","Type":"ContainerStarted","Data":"76b0a467798a00f720affa09e0c669da575ae4839da1222166511a60a133ab02"}
Sep 11 13:06:09.528003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:09.527944    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-ksck6" podStartSLOduration=2.454496567 podStartE2EDuration="3.527922633s" totalImagesPullingTime="1.013325244s" totalInitContainerRuntime="60.100822ms" isStatefulPod=true podCreationTimestamp="2026-09-11 13:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:06:09.52776069 +0000 UTC m=+84.047798036" watchObservedRunningTime="2026-09-11 13:06:09.527922633 +0000 UTC m=+84.047959979"
Sep 11 13:06:29.409831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:06:29.409793    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jz9hm"
Sep 11 13:09:45.843151 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:09:45.843122    2573 kubelet.go:1648] "Image garbage collection succeeded"
Sep 11 13:10:47.700046 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.699974    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"]
Sep 11 13:10:47.702860 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.702845    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.704969 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.704951    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
Sep 11 13:10:47.705630 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.705613    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-rtcjn\""
Sep 11 13:10:47.705773 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.705621    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
Sep 11 13:10:47.718008 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.717988    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"]
Sep 11 13:10:47.825634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.825612    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-679gg\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-kube-api-access-679gg\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.825755 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.825644    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.926788 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.926761    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-679gg\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-kube-api-access-679gg\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.926898 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.926795    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.938125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.938101    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:47.938909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:47.938890    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-679gg\" (UniqueName: \"kubernetes.io/projected/bc51d036-238b-41b8-b0ad-0039971aa8e3-kube-api-access-679gg\") pod \"cert-manager-cainjector-559cdd8c87-mpskp\" (UID: \"bc51d036-238b-41b8-b0ad-0039971aa8e3\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:48.011701 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:48.011682    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"
Sep 11 13:10:48.283653 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:48.283556    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-mpskp"]
Sep 11 13:10:48.287316 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:10:48.287289    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbc51d036_238b_41b8_b0ad_0039971aa8e3.slice/crio-0726efdf543a69690c1230c110e4aa356a9ab4c159844e8a126b76736f96a991 WatchSource:0}: Error finding container 0726efdf543a69690c1230c110e4aa356a9ab4c159844e8a126b76736f96a991: Status 404 returned error can't find the container with id 0726efdf543a69690c1230c110e4aa356a9ab4c159844e8a126b76736f96a991
Sep 11 13:10:48.289267 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:48.289251    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:10:48.350641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:48.350614    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp" event={"ID":"bc51d036-238b-41b8-b0ad-0039971aa8e3","Type":"ContainerStarted","Data":"0726efdf543a69690c1230c110e4aa356a9ab4c159844e8a126b76736f96a991"}
Sep 11 13:10:51.362693 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:51.362652    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp" event={"ID":"bc51d036-238b-41b8-b0ad-0039971aa8e3","Type":"ContainerStarted","Data":"52bf67240d090d12ccce19181583d3b53208909a31a994330986a01de40e4b52"}
Sep 11 13:10:51.385060 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:10:51.385006    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-mpskp" podStartSLOduration=1.50217285 podStartE2EDuration="4.384990294s" totalImagesPullingTime="2.882817444s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:10:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:10:51.384058249 +0000 UTC m=+365.904095594" watchObservedRunningTime="2026-09-11 13:10:51.384990294 +0000 UTC m=+365.905027639"
Sep 11 13:11:19.221190 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.221159    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"]
Sep 11 13:11:19.224034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.224015    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.230806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.230780    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-46rbt\""
Sep 11 13:11:19.230806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.230792    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
Sep 11 13:11:19.230806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.230785    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
Sep 11 13:11:19.231039 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.230818    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
Sep 11 13:11:19.231145 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.231128    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
Sep 11 13:11:19.253156 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.253125    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"]
Sep 11 13:11:19.330026 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.329992    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.330026 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.330027    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.330208 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.330055    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.330208 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.330080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qtjsl\" (UniqueName: \"kubernetes.io/projected/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-kube-api-access-qtjsl\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.330208 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.330149    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.430987    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.431025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.431041    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.431062    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.431088    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qtjsl\" (UniqueName: \"kubernetes.io/projected/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-kube-api-access-qtjsl\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.431475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.431453    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-manifests\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.433396 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.433373    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-tmp\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.433520 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.433502    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-apiservice-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.433690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.433669    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-webhook-cert\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.446329 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.446307    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qtjsl\" (UniqueName: \"kubernetes.io/projected/e80e8554-2a02-4f76-b8da-5f1f9b9f69ff-kube-api-access-qtjsl\") pod \"opendatahub-operator-controller-manager-5c4b86887d-szq2v\" (UID: \"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff\") " pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.533120 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.533097    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:19.830022 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:19.827666    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"]
Sep 11 13:11:19.833968 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:11:19.833945    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode80e8554_2a02_4f76_b8da_5f1f9b9f69ff.slice/crio-c532bd3e9fb990ff607be8478dab9a41237f737f978d964652a7d540772f0c94 WatchSource:0}: Error finding container c532bd3e9fb990ff607be8478dab9a41237f737f978d964652a7d540772f0c94: Status 404 returned error can't find the container with id c532bd3e9fb990ff607be8478dab9a41237f737f978d964652a7d540772f0c94
Sep 11 13:11:20.456811 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:20.456764    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v" event={"ID":"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff","Type":"ContainerStarted","Data":"c532bd3e9fb990ff607be8478dab9a41237f737f978d964652a7d540772f0c94"}
Sep 11 13:11:23.470697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:23.470663    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v" event={"ID":"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff","Type":"ContainerStarted","Data":"5ad62e4215d09fc324a41c073cf709737f2046ef1bfa161cb37e6b82de8f995f"}
Sep 11 13:11:24.475449 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:24.475364    2573 generic.go:356] "Generic (PLEG): container finished" podID="e80e8554-2a02-4f76-b8da-5f1f9b9f69ff" containerID="5ad62e4215d09fc324a41c073cf709737f2046ef1bfa161cb37e6b82de8f995f" exitCode=0
Sep 11 13:11:24.475449 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:24.475411    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v" event={"ID":"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff","Type":"ContainerDied","Data":"5ad62e4215d09fc324a41c073cf709737f2046ef1bfa161cb37e6b82de8f995f"}
Sep 11 13:11:25.480764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:25.480668    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v" event={"ID":"e80e8554-2a02-4f76-b8da-5f1f9b9f69ff","Type":"ContainerStarted","Data":"2be29cbee2f2e1b1df169928666b20696cf0010df1da67a8fa61ffc05223b2e3"}
Sep 11 13:11:25.481098 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:25.480803    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:25.508603 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:25.508553    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v" podStartSLOduration=1.849503131 podStartE2EDuration="6.508535009s" totalImagesPullingTime="3.600213269s" totalInitContainerRuntime="1.058818609s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:25.505893656 +0000 UTC m=+400.025931024" watchObservedRunningTime="2026-09-11 13:11:25.508535009 +0000 UTC m=+400.028572355"
Sep 11 13:11:36.486884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:36.486854    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-5c4b86887d-szq2v"
Sep 11 13:11:49.490725 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.490674    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d5cb"]
Sep 11 13:11:49.498408 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.498386    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.500915 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.500893    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-9q4bj\""
Sep 11 13:11:49.501867 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.501850    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\""
Sep 11 13:11:49.510739 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.510692    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d5cb"]
Sep 11 13:11:49.543209 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.543174    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.543383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.543232    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x5qld\" (UniqueName: \"kubernetes.io/projected/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-kube-api-access-x5qld\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.614866 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.614830    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"]
Sep 11 13:11:49.618570 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.618550    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.621445 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.621414    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\""
Sep 11 13:11:49.621848 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.621821    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
Sep 11 13:11:49.621967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.621848    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
Sep 11 13:11:49.621967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.621888    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\""
Sep 11 13:11:49.622143 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.622127    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-7wzq4\""
Sep 11 13:11:49.630962 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.630942    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"]
Sep 11 13:11:49.643720 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.643683    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.643830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.643746    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x5qld\" (UniqueName: \"kubernetes.io/projected/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-kube-api-access-x5qld\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.643830 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:11:49.643797    2573 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found
Sep 11 13:11:49.643906 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:11:49.643859    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert podName:d5ad7f4d-ff8d-4e35-891b-771687dabb8e nodeName:}" failed. No retries permitted until 2026-09-11 13:11:50.143845028 +0000 UTC m=+424.663882356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert") pod "odh-model-controller-7998c67dc4-2d5cb" (UID: "d5ad7f4d-ff8d-4e35-891b-771687dabb8e") : secret "odh-model-controller-webhook-cert" not found
Sep 11 13:11:49.659191 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.659173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x5qld\" (UniqueName: \"kubernetes.io/projected/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-kube-api-access-x5qld\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:49.744892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.744824    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875c6116-c68a-4aaa-9675-0509c02f5413-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.744892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.744871    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/875c6116-c68a-4aaa-9675-0509c02f5413-tmp\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.745043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.744920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7pbf4\" (UniqueName: \"kubernetes.io/projected/875c6116-c68a-4aaa-9675-0509c02f5413-kube-api-access-7pbf4\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.845517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.845474    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875c6116-c68a-4aaa-9675-0509c02f5413-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.845517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.845520    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/875c6116-c68a-4aaa-9675-0509c02f5413-tmp\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.845762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.845556    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7pbf4\" (UniqueName: \"kubernetes.io/projected/875c6116-c68a-4aaa-9675-0509c02f5413-kube-api-access-7pbf4\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.847734 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.847690    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/875c6116-c68a-4aaa-9675-0509c02f5413-tmp\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.848018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.848000    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/875c6116-c68a-4aaa-9675-0509c02f5413-tls-certs\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.856982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.856954    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7pbf4\" (UniqueName: \"kubernetes.io/projected/875c6116-c68a-4aaa-9675-0509c02f5413-kube-api-access-7pbf4\") pod \"kube-auth-proxy-74db4b4cbd-xd42v\" (UID: \"875c6116-c68a-4aaa-9675-0509c02f5413\") " pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:49.928952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:49.928914    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"
Sep 11 13:11:50.148029 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.147961    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:50.150217 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.150193    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d5ad7f4d-ff8d-4e35-891b-771687dabb8e-cert\") pod \"odh-model-controller-7998c67dc4-2d5cb\" (UID: \"d5ad7f4d-ff8d-4e35-891b-771687dabb8e\") " pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:50.200048 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.200023    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v"]
Sep 11 13:11:50.203052 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:11:50.203031    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod875c6116_c68a_4aaa_9675_0509c02f5413.slice/crio-3a6c61af3e815352865eca14fa15124af65fda1d270d61f20d1dd4554dc0bfbf WatchSource:0}: Error finding container 3a6c61af3e815352865eca14fa15124af65fda1d270d61f20d1dd4554dc0bfbf: Status 404 returned error can't find the container with id 3a6c61af3e815352865eca14fa15124af65fda1d270d61f20d1dd4554dc0bfbf
Sep 11 13:11:50.407982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.407916    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:50.565195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.565162    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v" event={"ID":"875c6116-c68a-4aaa-9675-0509c02f5413","Type":"ContainerStarted","Data":"3a6c61af3e815352865eca14fa15124af65fda1d270d61f20d1dd4554dc0bfbf"}
Sep 11 13:11:50.696475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:50.696398    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-7998c67dc4-2d5cb"]
Sep 11 13:11:50.700614 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:11:50.700586    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5ad7f4d_ff8d_4e35_891b_771687dabb8e.slice/crio-188671b313f5f7debc5df2f2f0e6338cb8270f69b4931a7c9265d4a2d1d112a8 WatchSource:0}: Error finding container 188671b313f5f7debc5df2f2f0e6338cb8270f69b4931a7c9265d4a2d1d112a8: Status 404 returned error can't find the container with id 188671b313f5f7debc5df2f2f0e6338cb8270f69b4931a7c9265d4a2d1d112a8
Sep 11 13:11:51.045625 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.045454    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:11:51.060642 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.060613    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.063817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.063487    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\""
Sep 11 13:11:51.063817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.063545    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-6nhx5\""
Sep 11 13:11:51.063817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.063496    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\""
Sep 11 13:11:51.064035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.063915    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\""
Sep 11 13:11:51.064669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.064639    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:11:51.074463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.074428    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:11:51.155986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.155945    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.155997    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.156029    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.156104    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.156221    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.156253    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.156354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.156289    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8dpx\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257058 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257030    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257071    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257141    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257198    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.257457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.257231    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8dpx\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.258408 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.258381    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.260579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.260433    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.260579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.260518    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.260579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.260542    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.261175 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.261139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.266325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.266302    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8dpx\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.266774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.266751    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token\") pod \"istiod-openshift-gateway-7b7995d5cd-b7d8c\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") " pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.374230 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.374148    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:51.573126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.573086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb" event={"ID":"d5ad7f4d-ff8d-4e35-891b-771687dabb8e","Type":"ContainerStarted","Data":"188671b313f5f7debc5df2f2f0e6338cb8270f69b4931a7c9265d4a2d1d112a8"}
Sep 11 13:11:51.680137 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:51.679993    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:11:52.521340 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:11:52.521308    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbcb23739_1f5d_4ec8_af6a_810fee6bbe77.slice/crio-4f7f1ee23730d97b4fadd83688871e666bae0b77174456b2111ad1b78f0a9011 WatchSource:0}: Error finding container 4f7f1ee23730d97b4fadd83688871e666bae0b77174456b2111ad1b78f0a9011: Status 404 returned error can't find the container with id 4f7f1ee23730d97b4fadd83688871e666bae0b77174456b2111ad1b78f0a9011
Sep 11 13:11:52.586811 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:52.586773    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" event={"ID":"bcb23739-1f5d-4ec8-af6a-810fee6bbe77","Type":"ContainerStarted","Data":"4f7f1ee23730d97b4fadd83688871e666bae0b77174456b2111ad1b78f0a9011"}
Sep 11 13:11:54.601323 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:54.601241    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb" event={"ID":"d5ad7f4d-ff8d-4e35-891b-771687dabb8e","Type":"ContainerStarted","Data":"81ae7517e6730d2332b4083b6b109cb2edb8193a3fb01a57b2562bf7903e8d8d"}
Sep 11 13:11:54.601323 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:54.601318    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:11:54.604187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:54.603755    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v" event={"ID":"875c6116-c68a-4aaa-9675-0509c02f5413","Type":"ContainerStarted","Data":"196c8027e0d3aaeebfe40076ea3bc823ac49f4837da1ace66ff9103bd0916a6d"}
Sep 11 13:11:54.623046 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:54.622986    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb" podStartSLOduration=1.911207585 podStartE2EDuration="5.622968658s" totalImagesPullingTime="3.711761073s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:54.622066363 +0000 UTC m=+429.142103708" watchObservedRunningTime="2026-09-11 13:11:54.622968658 +0000 UTC m=+429.143006031"
Sep 11 13:11:54.647223 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:54.647134    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-74db4b4cbd-xd42v" podStartSLOduration=1.449727054 podStartE2EDuration="5.647115958s" totalImagesPullingTime="4.197388904s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:54.645467389 +0000 UTC m=+429.165504738" watchObservedRunningTime="2026-09-11 13:11:54.647115958 +0000 UTC m=+429.167153304"
Sep 11 13:11:55.752879 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:55.752842    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:11:55.753114 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:55.752908    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:11:56.613911 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:56.613871    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" event={"ID":"bcb23739-1f5d-4ec8-af6a-810fee6bbe77","Type":"ContainerStarted","Data":"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"}
Sep 11 13:11:56.614103 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:56.614075    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:11:56.615528 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:56.615497    2573 patch_prober.go:28] interesting pod/istiod-openshift-gateway-7b7995d5cd-b7d8c container/discovery namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:11:56.615648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:56.615545    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerName="discovery" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:11:56.638677 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:56.638628    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" podStartSLOduration=2.409660298 podStartE2EDuration="5.638614073s" totalImagesPullingTime="3.228953775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:11:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:11:56.637747445 +0000 UTC m=+431.157784791" watchObservedRunningTime="2026-09-11 13:11:56.638614073 +0000 UTC m=+431.158651417"
Sep 11 13:11:57.618150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:11:57.618123    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:12:04.273105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.273071    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:12:04.273549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.273266    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerName="discovery" containerID="cri-o://85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b" gracePeriod=30
Sep 11 13:12:04.503801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.503770    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:12:04.561268 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561216    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561268 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561246    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l8dpx\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561279    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561309    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561335    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561360    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561383    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig\") pod \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\" (UID: \"bcb23739-1f5d-4ec8-af6a-810fee6bbe77\") "
Sep 11 13:12:04.561817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.561780    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:12:04.563624 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.563573    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs" (OuterVolumeSpecName: "local-certs") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:12:04.563770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.563729    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token" (OuterVolumeSpecName: "istio-token") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:12:04.564005 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.563987    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx" (OuterVolumeSpecName: "kube-api-access-l8dpx") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "kube-api-access-l8dpx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:12:04.564061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.564002    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:12:04.564246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.564225    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:12:04.564246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.564238    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts" (OuterVolumeSpecName: "cacerts") pod "bcb23739-1f5d-4ec8-af6a-810fee6bbe77" (UID: "bcb23739-1f5d-4ec8-af6a-810fee6bbe77"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:12:04.642914 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.642886    2573 generic.go:356] "Generic (PLEG): container finished" podID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerID="85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b" exitCode=0
Sep 11 13:12:04.643000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.642927    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"
Sep 11 13:12:04.643000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.642935    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" event={"ID":"bcb23739-1f5d-4ec8-af6a-810fee6bbe77","Type":"ContainerDied","Data":"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"}
Sep 11 13:12:04.643000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.642974    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c" event={"ID":"bcb23739-1f5d-4ec8-af6a-810fee6bbe77","Type":"ContainerDied","Data":"4f7f1ee23730d97b4fadd83688871e666bae0b77174456b2111ad1b78f0a9011"}
Sep 11 13:12:04.643000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.642998    2573 scope.go:117] "RemoveContainer" containerID="85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"
Sep 11 13:12:04.651027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.651008    2573 scope.go:117] "RemoveContainer" containerID="85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"
Sep 11 13:12:04.651401 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:12:04.651300    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b\": container with ID starting with 85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b not found: ID does not exist" containerID="85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"
Sep 11 13:12:04.651401 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.651334    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b"} err="failed to get container status \"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b\": rpc error: code = NotFound desc = could not find container \"85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b\": container with ID starting with 85ab870d52ec67ab516bf4bb5a49619136dacbe42a02bf1f2dd4b2a4aa98646b not found: ID does not exist"
Sep 11 13:12:04.662095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662078    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l8dpx\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-kube-api-access-l8dpx\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662098    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662108    2573 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-local-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662116    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-dns-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662124    2573 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-csr-ca-configmap\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662132    2573 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-istio-kubeconfig\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.662153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.662141    2573 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/bcb23739-1f5d-4ec8-af6a-810fee6bbe77-cacerts\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:12:04.749187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.749164    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:12:04.784955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:04.784934    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7b7995d5cd-b7d8c"]
Sep 11 13:12:05.610821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:05.610789    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-7998c67dc4-2d5cb"
Sep 11 13:12:05.970573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:12:05.970481    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" path="/var/lib/kubelet/pods/bcb23739-1f5d-4ec8-af6a-810fee6bbe77/volumes"
Sep 11 13:13:01.052668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.052632    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"]
Sep 11 13:13:01.053140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.052919    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerName="discovery"
Sep 11 13:13:01.053140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.052952    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerName="discovery"
Sep 11 13:13:01.053140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.053026    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bcb23739-1f5d-4ec8-af6a-810fee6bbe77" containerName="discovery"
Sep 11 13:13:01.055780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.055762    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.060309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.060280    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
Sep 11 13:13:01.060476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.060338    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-r6hnf\""
Sep 11 13:13:01.061054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.061029    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
Sep 11 13:13:01.077439 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.077414    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"]
Sep 11 13:13:01.135850 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.135821    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9lpmv\" (UniqueName: \"kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.135986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.135857    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.135986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.135878    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.145149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.145128    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"]
Sep 11 13:13:01.147209 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.147194    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.166571 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.166548    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"]
Sep 11 13:13:01.236264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236240    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2tj4q\" (UniqueName: \"kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.236377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236279    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9lpmv\" (UniqueName: \"kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.236377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236300    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.236377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236321    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.236377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236340    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.236377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236359    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.236763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236742    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.236805 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.236764    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.245940 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.245919    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"]
Sep 11 13:13:01.248090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.248076    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.251969 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.251950    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9lpmv\" (UniqueName: \"kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv\") pod \"da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") " pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.267399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.267378    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"]
Sep 11 13:13:01.337565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337514    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.337565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337549    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.337682 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337577    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r25lq\" (UniqueName: \"kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.337771 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337721    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.337824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337767    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2tj4q\" (UniqueName: \"kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.337824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337803    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.337894 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.337858    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.338046 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.338031    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.359073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.359044    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2tj4q\" (UniqueName: \"kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.364530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.364506    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:01.365754 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.365734    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"]
Sep 11 13:13:01.367909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.367892    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.384151 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.384133    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"]
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438592    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438653    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r25lq\" (UniqueName: \"kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438715    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438743    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438795    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6cl6\" (UniqueName: \"kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438826    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.438983    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.439254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.439218    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.447905 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.447873    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r25lq\" (UniqueName: \"kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq\") pod \"ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") " pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.455648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.455628    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:01.539125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.539101    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.539243 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.539146    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-v6cl6\" (UniqueName: \"kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.539243 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.539166    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.539520 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.539501    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.539556 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.539510    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.549671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.549653    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6cl6\" (UniqueName: \"kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6\") pod \"6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") " pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.561504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.561479    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:01.643277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.643251    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"]
Sep 11 13:13:01.643967 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:01.643946    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod55770797_ba97_44b2_a9aa_7a68afa0bd61.slice/crio-13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b WatchSource:0}: Error finding container 13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b: Status 404 returned error can't find the container with id 13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b
Sep 11 13:13:01.690519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.690496    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:01.748818 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.748151    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"]
Sep 11 13:13:01.839000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.838939    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"]
Sep 11 13:13:01.841156 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:01.841125    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9e8d051_bd2e_4f1b_83a6_61e98f4da74b.slice/crio-dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3 WatchSource:0}: Error finding container dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3: Status 404 returned error can't find the container with id dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3
Sep 11 13:13:01.841551 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.841521    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6" event={"ID":"55770797-ba97-44b2-a9aa-7a68afa0bd61","Type":"ContainerStarted","Data":"13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b"}
Sep 11 13:13:01.842869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.842839    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6" event={"ID":"05e1295f-1ac2-4cbf-9202-f75bddc02128","Type":"ContainerStarted","Data":"0d2c5caffcb0277421cf2f837af07c22e3552b6793ab71aa6c9fcbb96f9e4b62"}
Sep 11 13:13:01.976355 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:01.976333    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"]
Sep 11 13:13:01.978241 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:01.978216    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod52e81fab_6ba5_4202_886d_2516ad41dded.slice/crio-a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764 WatchSource:0}: Error finding container a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764: Status 404 returned error can't find the container with id a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764
Sep 11 13:13:02.850210 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:02.850167    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8" event={"ID":"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b","Type":"ContainerStarted","Data":"dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3"}
Sep 11 13:13:02.851635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:02.851588    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs" event={"ID":"52e81fab-6ba5-4202-886d-2516ad41dded","Type":"ContainerStarted","Data":"a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764"}
Sep 11 13:13:08.886844 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.886807    2573 generic.go:356] "Generic (PLEG): container finished" podID="52e81fab-6ba5-4202-886d-2516ad41dded" containerID="d1d6350f9ab5a3615fc90d51dc5f154837ef692802f259ff1b57e291487e8589" exitCode=0
Sep 11 13:13:08.887187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.886901    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs" event={"ID":"52e81fab-6ba5-4202-886d-2516ad41dded","Type":"ContainerDied","Data":"d1d6350f9ab5a3615fc90d51dc5f154837ef692802f259ff1b57e291487e8589"}
Sep 11 13:13:08.888570 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.888548    2573 generic.go:356] "Generic (PLEG): container finished" podID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerID="902294c43e1fa2c8d9ada3e01a8c0f5e7a5022f9c2b056a0c3d763151fdd99b2" exitCode=0
Sep 11 13:13:08.888649 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.888628    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6" event={"ID":"55770797-ba97-44b2-a9aa-7a68afa0bd61","Type":"ContainerDied","Data":"902294c43e1fa2c8d9ada3e01a8c0f5e7a5022f9c2b056a0c3d763151fdd99b2"}
Sep 11 13:13:08.890137 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.890108    2573 generic.go:356] "Generic (PLEG): container finished" podID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerID="7bd97ecde5ef010d47506f38673ed3ce525af909091c1bc695dba8382ce5721e" exitCode=0
Sep 11 13:13:08.890231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.890175    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6" event={"ID":"05e1295f-1ac2-4cbf-9202-f75bddc02128","Type":"ContainerDied","Data":"7bd97ecde5ef010d47506f38673ed3ce525af909091c1bc695dba8382ce5721e"}
Sep 11 13:13:08.891852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.891832    2573 generic.go:356] "Generic (PLEG): container finished" podID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerID="60b5223cf61c594fb18687a849e68be912ce63cb390343cceecd949f1e7e4eff" exitCode=0
Sep 11 13:13:08.891948 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:08.891874    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8" event={"ID":"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b","Type":"ContainerDied","Data":"60b5223cf61c594fb18687a849e68be912ce63cb390343cceecd949f1e7e4eff"}
Sep 11 13:13:09.897957 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.897917    2573 generic.go:356] "Generic (PLEG): container finished" podID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerID="9502540d235ea311b8d7f3606d3c71da93795d3e896c39b0cc1a03cbeec35906" exitCode=0
Sep 11 13:13:09.898310 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.897979    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6" event={"ID":"05e1295f-1ac2-4cbf-9202-f75bddc02128","Type":"ContainerDied","Data":"9502540d235ea311b8d7f3606d3c71da93795d3e896c39b0cc1a03cbeec35906"}
Sep 11 13:13:09.899955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.899932    2573 generic.go:356] "Generic (PLEG): container finished" podID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerID="cd7faec6b9332692092f204e7d9ed02148216aae2db42097e4ad2084d86a86c8" exitCode=0
Sep 11 13:13:09.900057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.899980    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8" event={"ID":"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b","Type":"ContainerDied","Data":"cd7faec6b9332692092f204e7d9ed02148216aae2db42097e4ad2084d86a86c8"}
Sep 11 13:13:09.902016 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.901997    2573 generic.go:356] "Generic (PLEG): container finished" podID="52e81fab-6ba5-4202-886d-2516ad41dded" containerID="0314af1ba0d1b5a31f85a7114e5a0953267fa8fd31363d9747b6cd591f0ae8f5" exitCode=0
Sep 11 13:13:09.902099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.902074    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs" event={"ID":"52e81fab-6ba5-4202-886d-2516ad41dded","Type":"ContainerDied","Data":"0314af1ba0d1b5a31f85a7114e5a0953267fa8fd31363d9747b6cd591f0ae8f5"}
Sep 11 13:13:09.904283 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.904260    2573 generic.go:356] "Generic (PLEG): container finished" podID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerID="6815cac5a6605460c420fa37e4feee6e1746ec6355b8b6759939d68c1efe63bf" exitCode=0
Sep 11 13:13:09.904384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:09.904338    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6" event={"ID":"55770797-ba97-44b2-a9aa-7a68afa0bd61","Type":"ContainerDied","Data":"6815cac5a6605460c420fa37e4feee6e1746ec6355b8b6759939d68c1efe63bf"}
Sep 11 13:13:17.976376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.976347    2573 generic.go:356] "Generic (PLEG): container finished" podID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerID="bb2e23eb12e98081b07b07a1b7ccfbe5b7570f261f044eddcea2dab8b647c34c" exitCode=0
Sep 11 13:13:17.976692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.976423    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6" event={"ID":"05e1295f-1ac2-4cbf-9202-f75bddc02128","Type":"ContainerDied","Data":"bb2e23eb12e98081b07b07a1b7ccfbe5b7570f261f044eddcea2dab8b647c34c"}
Sep 11 13:13:17.978390 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.978368    2573 generic.go:356] "Generic (PLEG): container finished" podID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerID="3f8e05f44d254432d44d8471c5cc0f0ae449d4afde024e62fc464f2c2b46ec1f" exitCode=0
Sep 11 13:13:17.978476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.978447    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8" event={"ID":"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b","Type":"ContainerDied","Data":"3f8e05f44d254432d44d8471c5cc0f0ae449d4afde024e62fc464f2c2b46ec1f"}
Sep 11 13:13:17.980319 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.980303    2573 generic.go:356] "Generic (PLEG): container finished" podID="52e81fab-6ba5-4202-886d-2516ad41dded" containerID="0a5f8768774cf8f849b8ed861accb69c7a4c5d6258604d7f8c11960681abc23b" exitCode=0
Sep 11 13:13:17.980408 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.980358    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs" event={"ID":"52e81fab-6ba5-4202-886d-2516ad41dded","Type":"ContainerDied","Data":"0a5f8768774cf8f849b8ed861accb69c7a4c5d6258604d7f8c11960681abc23b"}
Sep 11 13:13:17.982212 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.982195    2573 generic.go:356] "Generic (PLEG): container finished" podID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerID="f9b9d9a80f1f75ae03e00803fe60ae03de46e8bfbb935d854a9dafd72383837b" exitCode=0
Sep 11 13:13:17.982287 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:17.982251    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6" event={"ID":"55770797-ba97-44b2-a9aa-7a68afa0bd61","Type":"ContainerDied","Data":"f9b9d9a80f1f75ae03e00803fe60ae03de46e8bfbb935d854a9dafd72383837b"}
Sep 11 13:13:19.120173 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.120151    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:19.144619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.144598    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:19.181261 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.181243    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:19.187359 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187341    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:19.187558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187546    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9lpmv\" (UniqueName: \"kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv\") pod \"55770797-ba97-44b2-a9aa-7a68afa0bd61\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") "
Sep 11 13:13:19.187599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187582    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util\") pod \"55770797-ba97-44b2-a9aa-7a68afa0bd61\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") "
Sep 11 13:13:19.187645 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187612    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2tj4q\" (UniqueName: \"kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q\") pod \"05e1295f-1ac2-4cbf-9202-f75bddc02128\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") "
Sep 11 13:13:19.187701 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187651    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util\") pod \"05e1295f-1ac2-4cbf-9202-f75bddc02128\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") "
Sep 11 13:13:19.187701 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187677    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle\") pod \"55770797-ba97-44b2-a9aa-7a68afa0bd61\" (UID: \"55770797-ba97-44b2-a9aa-7a68afa0bd61\") "
Sep 11 13:13:19.187833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.187749    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle\") pod \"05e1295f-1ac2-4cbf-9202-f75bddc02128\" (UID: \"05e1295f-1ac2-4cbf-9202-f75bddc02128\") "
Sep 11 13:13:19.188246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.188217    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle" (OuterVolumeSpecName: "bundle") pod "55770797-ba97-44b2-a9aa-7a68afa0bd61" (UID: "55770797-ba97-44b2-a9aa-7a68afa0bd61"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.188357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.188302    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle" (OuterVolumeSpecName: "bundle") pod "05e1295f-1ac2-4cbf-9202-f75bddc02128" (UID: "05e1295f-1ac2-4cbf-9202-f75bddc02128"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.190040 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.190019    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q" (OuterVolumeSpecName: "kube-api-access-2tj4q") pod "05e1295f-1ac2-4cbf-9202-f75bddc02128" (UID: "05e1295f-1ac2-4cbf-9202-f75bddc02128"). InnerVolumeSpecName "kube-api-access-2tj4q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:19.190139 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.190119    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv" (OuterVolumeSpecName: "kube-api-access-9lpmv") pod "55770797-ba97-44b2-a9aa-7a68afa0bd61" (UID: "55770797-ba97-44b2-a9aa-7a68afa0bd61"). InnerVolumeSpecName "kube-api-access-9lpmv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:19.191875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.191854    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util" (OuterVolumeSpecName: "util") pod "05e1295f-1ac2-4cbf-9202-f75bddc02128" (UID: "05e1295f-1ac2-4cbf-9202-f75bddc02128"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.192807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.192787    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util" (OuterVolumeSpecName: "util") pod "55770797-ba97-44b2-a9aa-7a68afa0bd61" (UID: "55770797-ba97-44b2-a9aa-7a68afa0bd61"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.288400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288369    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util\") pod \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") "
Sep 11 13:13:19.288569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288410    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle\") pod \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") "
Sep 11 13:13:19.288569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288437    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r25lq\" (UniqueName: \"kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq\") pod \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\" (UID: \"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b\") "
Sep 11 13:13:19.288569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288466    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle\") pod \"52e81fab-6ba5-4202-886d-2516ad41dded\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") "
Sep 11 13:13:19.288569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288510    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util\") pod \"52e81fab-6ba5-4202-886d-2516ad41dded\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") "
Sep 11 13:13:19.288569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288558    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-v6cl6\" (UniqueName: \"kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6\") pod \"52e81fab-6ba5-4202-886d-2516ad41dded\" (UID: \"52e81fab-6ba5-4202-886d-2516ad41dded\") "
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288731    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2tj4q\" (UniqueName: \"kubernetes.io/projected/05e1295f-1ac2-4cbf-9202-f75bddc02128-kube-api-access-2tj4q\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288749    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-util\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288762    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-bundle\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288773    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/05e1295f-1ac2-4cbf-9202-f75bddc02128-bundle\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288785    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9lpmv\" (UniqueName: \"kubernetes.io/projected/55770797-ba97-44b2-a9aa-7a68afa0bd61-kube-api-access-9lpmv\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.288857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288798    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/55770797-ba97-44b2-a9aa-7a68afa0bd61-util\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.289123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.288986    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle" (OuterVolumeSpecName: "bundle") pod "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" (UID: "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.289266 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.289223    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle" (OuterVolumeSpecName: "bundle") pod "52e81fab-6ba5-4202-886d-2516ad41dded" (UID: "52e81fab-6ba5-4202-886d-2516ad41dded"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.290620 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.290591    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6" (OuterVolumeSpecName: "kube-api-access-v6cl6") pod "52e81fab-6ba5-4202-886d-2516ad41dded" (UID: "52e81fab-6ba5-4202-886d-2516ad41dded"). InnerVolumeSpecName "kube-api-access-v6cl6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:19.290700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.290612    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq" (OuterVolumeSpecName: "kube-api-access-r25lq") pod "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" (UID: "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b"). InnerVolumeSpecName "kube-api-access-r25lq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:13:19.293054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.293004    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util" (OuterVolumeSpecName: "util") pod "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" (UID: "f9e8d051-bd2e-4f1b-83a6-61e98f4da74b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.293337 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.293319    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util" (OuterVolumeSpecName: "util") pod "52e81fab-6ba5-4202-886d-2516ad41dded" (UID: "52e81fab-6ba5-4202-886d-2516ad41dded"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:13:19.389933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389902    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-util\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.389933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389931    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-v6cl6\" (UniqueName: \"kubernetes.io/projected/52e81fab-6ba5-4202-886d-2516ad41dded-kube-api-access-v6cl6\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.389933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389941    2573 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-util\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.390074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389949    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-bundle\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.390074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389957    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r25lq\" (UniqueName: \"kubernetes.io/projected/f9e8d051-bd2e-4f1b-83a6-61e98f4da74b-kube-api-access-r25lq\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.390074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.389966    2573 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/52e81fab-6ba5-4202-886d-2516ad41dded-bundle\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:13:19.991601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.991567    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6" event={"ID":"05e1295f-1ac2-4cbf-9202-f75bddc02128","Type":"ContainerDied","Data":"0d2c5caffcb0277421cf2f837af07c22e3552b6793ab71aa6c9fcbb96f9e4b62"}
Sep 11 13:13:19.991601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.991596    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4cqzbg6"
Sep 11 13:13:19.991764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.991616    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="0d2c5caffcb0277421cf2f837af07c22e3552b6793ab71aa6c9fcbb96f9e4b62"
Sep 11 13:13:19.993740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.993699    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8" event={"ID":"f9e8d051-bd2e-4f1b-83a6-61e98f4da74b","Type":"ContainerDied","Data":"dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3"}
Sep 11 13:13:19.993833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.993755    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="dc35c83b479a68db9ff8c4602c6d3b5be119f06132dbdae9a959d438feb5c6c3"
Sep 11 13:13:19.993833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.993822    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/ea579a99e39e9baa67b0fe2e51b0fa4ef69da3ae0011f15b849816c0c6p77n8"
Sep 11 13:13:19.995581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.995561    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs" event={"ID":"52e81fab-6ba5-4202-886d-2516ad41dded","Type":"ContainerDied","Data":"a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764"}
Sep 11 13:13:19.995665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.995579    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/6541d93aaee335d6227a8778ab4e840bbe7f5273fe8e69d6bf3410847a2bbvs"
Sep 11 13:13:19.995665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.995597    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a62f114649ba79281175f3c144789c770c53c85363f995f28c1f23b03def5764"
Sep 11 13:13:19.997760 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.997737    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6" event={"ID":"55770797-ba97-44b2-a9aa-7a68afa0bd61","Type":"ContainerDied","Data":"13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b"}
Sep 11 13:13:19.997845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.997776    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="13010612d377fedd5ee14475af2c42fbdca76e0a8d5710a60aaeddee672ad27b"
Sep 11 13:13:19.997845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:19.997841    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/da112bd609bff87f7bacd2ba941225d7cff15c41059c552dee22e264d7tgxn6"
Sep 11 13:13:31.285328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285286    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"]
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285642    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285658    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285669    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285677    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285689    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285697    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285731    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285741    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285751    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285759    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285769    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285778    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285790    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285798    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285810    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285818    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285829    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285837    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285849    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285857    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="extract"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285867    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285875    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="pull"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285885    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="util"
Sep 11 13:13:31.285916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285893    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="util"
Sep 11 13:13:31.287126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285979    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="55770797-ba97-44b2-a9aa-7a68afa0bd61" containerName="extract"
Sep 11 13:13:31.287126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.285990    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="05e1295f-1ac2-4cbf-9202-f75bddc02128" containerName="extract"
Sep 11 13:13:31.287126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.286000    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="52e81fab-6ba5-4202-886d-2516ad41dded" containerName="extract"
Sep 11 13:13:31.287126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.286009    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f9e8d051-bd2e-4f1b-83a6-61e98f4da74b" containerName="extract"
Sep 11 13:13:31.289366 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.289345    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:31.292542 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.292511    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\""
Sep 11 13:13:31.292646 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.292597    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-2fml5\""
Sep 11 13:13:31.292886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.292870    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
Sep 11 13:13:31.292886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.292878    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
Sep 11 13:13:31.305573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.305553    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"]
Sep 11 13:13:31.364517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.364486    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gv7n\" (UniqueName: \"kubernetes.io/projected/20aa07aa-5187-4777-b761-d7b3287a450f-kube-api-access-7gv7n\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vb4s\" (UID: \"20aa07aa-5187-4777-b761-d7b3287a450f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:31.465535 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.465509    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7gv7n\" (UniqueName: \"kubernetes.io/projected/20aa07aa-5187-4777-b761-d7b3287a450f-kube-api-access-7gv7n\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vb4s\" (UID: \"20aa07aa-5187-4777-b761-d7b3287a450f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:31.477853 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.477822    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gv7n\" (UniqueName: \"kubernetes.io/projected/20aa07aa-5187-4777-b761-d7b3287a450f-kube-api-access-7gv7n\") pod \"dns-operator-controller-manager-7b84cd7c9c-5vb4s\" (UID: \"20aa07aa-5187-4777-b761-d7b3287a450f\") " pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:31.599671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.599610    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:31.903114 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:31.903091    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"]
Sep 11 13:13:31.904865 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:31.904836    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20aa07aa_5187_4777_b761_d7b3287a450f.slice/crio-4d784c5d9f448b9de2e12490dca646b7a06b6cf17e3e1fad95d7cd38785a06bf WatchSource:0}: Error finding container 4d784c5d9f448b9de2e12490dca646b7a06b6cf17e3e1fad95d7cd38785a06bf: Status 404 returned error can't find the container with id 4d784c5d9f448b9de2e12490dca646b7a06b6cf17e3e1fad95d7cd38785a06bf
Sep 11 13:13:32.045931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:32.045897    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s" event={"ID":"20aa07aa-5187-4777-b761-d7b3287a450f","Type":"ContainerStarted","Data":"4d784c5d9f448b9de2e12490dca646b7a06b6cf17e3e1fad95d7cd38785a06bf"}
Sep 11 13:13:35.423550 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.423521    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vsqmw"]
Sep 11 13:13:35.426119 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.426103    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:35.430024 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.430004    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-gkmjt\""
Sep 11 13:13:35.455981 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.455961    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vsqmw"]
Sep 11 13:13:35.493253 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.493230    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmhml\" (UniqueName: \"kubernetes.io/projected/ed7d162a-cc13-4b84-8c33-7f44f6fefb03-kube-api-access-mmhml\") pod \"authorino-operator-76b478bd78-vsqmw\" (UID: \"ed7d162a-cc13-4b84-8c33-7f44f6fefb03\") " pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:35.594522 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.594498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmhml\" (UniqueName: \"kubernetes.io/projected/ed7d162a-cc13-4b84-8c33-7f44f6fefb03-kube-api-access-mmhml\") pod \"authorino-operator-76b478bd78-vsqmw\" (UID: \"ed7d162a-cc13-4b84-8c33-7f44f6fefb03\") " pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:35.635721 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.635688    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmhml\" (UniqueName: \"kubernetes.io/projected/ed7d162a-cc13-4b84-8c33-7f44f6fefb03-kube-api-access-mmhml\") pod \"authorino-operator-76b478bd78-vsqmw\" (UID: \"ed7d162a-cc13-4b84-8c33-7f44f6fefb03\") " pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:35.735686 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:35.735640    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:36.038046 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:36.038021    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-76b478bd78-vsqmw"]
Sep 11 13:13:36.040719 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:36.040674    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded7d162a_cc13_4b84_8c33_7f44f6fefb03.slice/crio-9c665ab9264f92a7dc22ea98aed90cd17c0acdc38e5573148bb5b982bdabb760 WatchSource:0}: Error finding container 9c665ab9264f92a7dc22ea98aed90cd17c0acdc38e5573148bb5b982bdabb760: Status 404 returned error can't find the container with id 9c665ab9264f92a7dc22ea98aed90cd17c0acdc38e5573148bb5b982bdabb760
Sep 11 13:13:36.062080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:36.062050    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw" event={"ID":"ed7d162a-cc13-4b84-8c33-7f44f6fefb03","Type":"ContainerStarted","Data":"9c665ab9264f92a7dc22ea98aed90cd17c0acdc38e5573148bb5b982bdabb760"}
Sep 11 13:13:39.454632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.454543    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"]
Sep 11 13:13:39.458233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.458210    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:39.461838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.461817    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-fb4k8\""
Sep 11 13:13:39.486154 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.486130    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"]
Sep 11 13:13:39.522812 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.522789    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6fbpp\" (UniqueName: \"kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp\") pod \"limitador-operator-controller-manager-74495c69f8-pvqzg\" (UID: \"0e2914cb-1d72-48a5-a127-9f46e43566f3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:39.623806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.623776    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6fbpp\" (UniqueName: \"kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp\") pod \"limitador-operator-controller-manager-74495c69f8-pvqzg\" (UID: \"0e2914cb-1d72-48a5-a127-9f46e43566f3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:39.636827 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.636800    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6fbpp\" (UniqueName: \"kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp\") pod \"limitador-operator-controller-manager-74495c69f8-pvqzg\" (UID: \"0e2914cb-1d72-48a5-a127-9f46e43566f3\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:39.769623 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:39.769590    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:40.271425 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:40.271288    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"]
Sep 11 13:13:40.273997 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:40.273941    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e2914cb_1d72_48a5_a127_9f46e43566f3.slice/crio-c4c8628523ab2c523b020d36c5425107676239bd325204789116ad457014fc0e WatchSource:0}: Error finding container c4c8628523ab2c523b020d36c5425107676239bd325204789116ad457014fc0e: Status 404 returned error can't find the container with id c4c8628523ab2c523b020d36c5425107676239bd325204789116ad457014fc0e
Sep 11 13:13:41.086316 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.086282    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s" event={"ID":"20aa07aa-5187-4777-b761-d7b3287a450f","Type":"ContainerStarted","Data":"a2d4a20d64bc9b8332f3b22123e99c087a27b877877ed2fef196d0e9d3fde18f"}
Sep 11 13:13:41.086798 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.086488    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:41.087772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.087747    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg" event={"ID":"0e2914cb-1d72-48a5-a127-9f46e43566f3","Type":"ContainerStarted","Data":"c4c8628523ab2c523b020d36c5425107676239bd325204789116ad457014fc0e"}
Sep 11 13:13:41.089534 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.089506    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw" event={"ID":"ed7d162a-cc13-4b84-8c33-7f44f6fefb03","Type":"ContainerStarted","Data":"870545932c2cc577ec12e521e31030eae4da8002ad6e28cc1a8171680acd932a"}
Sep 11 13:13:41.089666 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.089644    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:41.129269 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:41.129224    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s" podStartSLOduration=2.006072312 podStartE2EDuration="10.129206997s" totalImagesPullingTime="8.123134685s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:41.128839217 +0000 UTC m=+535.648876564" watchObservedRunningTime="2026-09-11 13:13:41.129206997 +0000 UTC m=+535.649244342"
Sep 11 13:13:42.095407 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:42.095364    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg" event={"ID":"0e2914cb-1d72-48a5-a127-9f46e43566f3","Type":"ContainerStarted","Data":"d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1"}
Sep 11 13:13:42.095860 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:42.095503    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:42.213919 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:42.213880    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw" podStartSLOduration=2.614411522 podStartE2EDuration="7.213866836s" totalImagesPullingTime="4.599455314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:41.178598239 +0000 UTC m=+535.698635582" watchObservedRunningTime="2026-09-11 13:13:42.213866836 +0000 UTC m=+536.733904181"
Sep 11 13:13:42.214200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:42.214174    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg" podStartSLOduration=1.841440635 podStartE2EDuration="3.21416812s" totalImagesPullingTime="1.372727485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:42.193338326 +0000 UTC m=+536.713375671" watchObservedRunningTime="2026-09-11 13:13:42.21416812 +0000 UTC m=+536.734205463"
Sep 11 13:13:45.900181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.900151    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"]
Sep 11 13:13:45.902472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.902455    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:45.908241 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.908219    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-k5fmn\""
Sep 11 13:13:45.921032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.921010    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"]
Sep 11 13:13:45.972635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.972611    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbglq\" (UniqueName: \"kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:45.972746 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:45.972644    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.073779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.073759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nbglq\" (UniqueName: \"kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.073870 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.073788    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.074065 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.074050    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.088910 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.088883    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbglq\" (UniqueName: \"kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq\") pod \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.212081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.212025    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:46.499992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:46.499961    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"]
Sep 11 13:13:46.503603 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:46.503579    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode680f714_8be8_4c3c_9ba9_407d103117d7.slice/crio-6885032953ed2376ea88771cd094870c47a4fee1978a267e8c1761a573190219 WatchSource:0}: Error finding container 6885032953ed2376ea88771cd094870c47a4fee1978a267e8c1761a573190219: Status 404 returned error can't find the container with id 6885032953ed2376ea88771cd094870c47a4fee1978a267e8c1761a573190219
Sep 11 13:13:47.117780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:47.117749    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8" event={"ID":"e680f714-8be8-4c3c-9ba9-407d103117d7","Type":"ContainerStarted","Data":"6885032953ed2376ea88771cd094870c47a4fee1978a267e8c1761a573190219"}
Sep 11 13:13:50.133865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:50.133827    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8" event={"ID":"e680f714-8be8-4c3c-9ba9-407d103117d7","Type":"ContainerStarted","Data":"3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309"}
Sep 11 13:13:50.134224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:50.133958    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:13:50.157533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:50.157483    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8" podStartSLOduration=1.753801763 podStartE2EDuration="5.157467553s" totalImagesPullingTime="3.40366579s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:50.15564315 +0000 UTC m=+544.675680510" watchObservedRunningTime="2026-09-11 13:13:50.157467553 +0000 UTC m=+544.677504898"
Sep 11 13:13:52.098222 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.098193    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-76b478bd78-vsqmw"
Sep 11 13:13:52.098696 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.098298    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-7b84cd7c9c-5vb4s"
Sep 11 13:13:52.978693 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.978657    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"]
Sep 11 13:13:52.981519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.981501    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:52.984113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.984090    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
Sep 11 13:13:52.984280 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.984261    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
Sep 11 13:13:52.984350 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:52.984335    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-dfjd9\""
Sep 11 13:13:53.001990 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.001968    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"]
Sep 11 13:13:53.101916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.101894    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:13:53.122962 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.122937    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5shfz\" (UniqueName: \"kubernetes.io/projected/d62b639f-8df9-4b08-85ec-776ed90937f9-kube-api-access-5shfz\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.123064 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.123001    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d62b639f-8df9-4b08-85ec-776ed90937f9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.123064 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.123029    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d62b639f-8df9-4b08-85ec-776ed90937f9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.223778 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.223745    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d62b639f-8df9-4b08-85ec-776ed90937f9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.223946 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.223784    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d62b639f-8df9-4b08-85ec-776ed90937f9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.223946 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.223823    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5shfz\" (UniqueName: \"kubernetes.io/projected/d62b639f-8df9-4b08-85ec-776ed90937f9-kube-api-access-5shfz\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.224479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.224457    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/d62b639f-8df9-4b08-85ec-776ed90937f9-nginx-conf\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.226060 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.226038    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/d62b639f-8df9-4b08-85ec-776ed90937f9-plugin-serving-cert\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.237939 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.237913    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5shfz\" (UniqueName: \"kubernetes.io/projected/d62b639f-8df9-4b08-85ec-776ed90937f9-kube-api-access-5shfz\") pod \"kuadrant-console-plugin-fbdd4f445-9d429\" (UID: \"d62b639f-8df9-4b08-85ec-776ed90937f9\") " pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.291776 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.291746    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"
Sep 11 13:13:53.566823 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:53.566797    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429"]
Sep 11 13:13:53.568085 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:13:53.568052    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd62b639f_8df9_4b08_85ec_776ed90937f9.slice/crio-74c1e1b39941659a4757142d64cc8c79454a4f611aa3053aacdd1e5ff05370c9 WatchSource:0}: Error finding container 74c1e1b39941659a4757142d64cc8c79454a4f611aa3053aacdd1e5ff05370c9: Status 404 returned error can't find the container with id 74c1e1b39941659a4757142d64cc8c79454a4f611aa3053aacdd1e5ff05370c9
Sep 11 13:13:54.153484 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:54.153450    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429" event={"ID":"d62b639f-8df9-4b08-85ec-776ed90937f9","Type":"ContainerStarted","Data":"74c1e1b39941659a4757142d64cc8c79454a4f611aa3053aacdd1e5ff05370c9"}
Sep 11 13:13:59.178938 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:59.178894    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429" event={"ID":"d62b639f-8df9-4b08-85ec-776ed90937f9","Type":"ContainerStarted","Data":"6cfeb50e654fec548eaa71c3e32b708db8fa82d5b5ae1a6e92cd477580754818"}
Sep 11 13:13:59.213037 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:13:59.212986    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-fbdd4f445-9d429" podStartSLOduration=2.395878637 podStartE2EDuration="7.212971478s" totalImagesPullingTime="4.817092841s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:13:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:13:59.212772964 +0000 UTC m=+553.732810309" watchObservedRunningTime="2026-09-11 13:13:59.212971478 +0000 UTC m=+553.733008823"
Sep 11 13:14:01.142147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:01.142112    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:14:03.296601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.296566    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"]
Sep 11 13:14:03.297009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.296747    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" containerName="manager" containerID="cri-o://3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309" gracePeriod=2
Sep 11 13:14:03.307744 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.304931    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"]
Sep 11 13:14:03.324380 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.324356    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"]
Sep 11 13:14:03.324735 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.324680    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" containerName="manager" containerID="cri-o://d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1" gracePeriod=2
Sep 11 13:14:03.327635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.327611    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"]
Sep 11 13:14:03.361034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361004    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:03.361293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361277    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" containerName="manager"
Sep 11 13:14:03.361293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361292    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" containerName="manager"
Sep 11 13:14:03.361414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361308    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" containerName="manager"
Sep 11 13:14:03.361414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361314    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" containerName="manager"
Sep 11 13:14:03.361414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361363    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" containerName="manager"
Sep 11 13:14:03.361414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.361371    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" containerName="manager"
Sep 11 13:14:03.364893 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.364873    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.371360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.371334    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"]
Sep 11 13:14:03.373698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.373680    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:03.381818 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.381794    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:03.389203 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.389179    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"]
Sep 11 13:14:03.500253 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.500127    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbs88\" (UniqueName: \"kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.500253 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.500179    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4mk9\" (UniqueName: \"kubernetes.io/projected/146648ff-6004-4daf-b36f-697f16fd35c8-kube-api-access-z4mk9\") pod \"limitador-operator-controller-manager-74495c69f8-9p4mn\" (UID: \"146648ff-6004-4daf-b36f-697f16fd35c8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:03.500454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.500266    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.561733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.561695    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:14:03.564393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.564375    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:14:03.601326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.601300    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbs88\" (UniqueName: \"kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.601404 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.601331    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4mk9\" (UniqueName: \"kubernetes.io/projected/146648ff-6004-4daf-b36f-697f16fd35c8-kube-api-access-z4mk9\") pod \"limitador-operator-controller-manager-74495c69f8-9p4mn\" (UID: \"146648ff-6004-4daf-b36f-697f16fd35c8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:03.601404 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.601361    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.601634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.601619    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.615013 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.614987    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbs88\" (UniqueName: \"kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88\") pod \"kuadrant-operator-controller-manager-76f8f686bf-wz7wl\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") " pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.615089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.615051    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4mk9\" (UniqueName: \"kubernetes.io/projected/146648ff-6004-4daf-b36f-697f16fd35c8-kube-api-access-z4mk9\") pod \"limitador-operator-controller-manager-74495c69f8-9p4mn\" (UID: \"146648ff-6004-4daf-b36f-697f16fd35c8\") " pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:03.617451 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:03.617429    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:14:03.619316 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:03.619294    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pvqzg\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:14:03.669907 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:03.669875    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-76f8f686bf-t2sm8\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:14:03.671835 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:03.671812    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-74495c69f8-pvqzg\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:14:03.702092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.702060    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6fbpp\" (UniqueName: \"kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp\") pod \"0e2914cb-1d72-48a5-a127-9f46e43566f3\" (UID: \"0e2914cb-1d72-48a5-a127-9f46e43566f3\") "
Sep 11 13:14:03.702166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.702102    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume\") pod \"e680f714-8be8-4c3c-9ba9-407d103117d7\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") "
Sep 11 13:14:03.702166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.702141    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbglq\" (UniqueName: \"kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq\") pod \"e680f714-8be8-4c3c-9ba9-407d103117d7\" (UID: \"e680f714-8be8-4c3c-9ba9-407d103117d7\") "
Sep 11 13:14:03.702618 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.702595    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e680f714-8be8-4c3c-9ba9-407d103117d7" (UID: "e680f714-8be8-4c3c-9ba9-407d103117d7"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:14:03.704176 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.704152    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp" (OuterVolumeSpecName: "kube-api-access-6fbpp") pod "0e2914cb-1d72-48a5-a127-9f46e43566f3" (UID: "0e2914cb-1d72-48a5-a127-9f46e43566f3"). InnerVolumeSpecName "kube-api-access-6fbpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:03.704250 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.704192    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq" (OuterVolumeSpecName: "kube-api-access-nbglq") pod "e680f714-8be8-4c3c-9ba9-407d103117d7" (UID: "e680f714-8be8-4c3c-9ba9-407d103117d7"). InnerVolumeSpecName "kube-api-access-nbglq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:03.738391 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.738368    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:03.744780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.744763    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:03.803375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.803315    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbglq\" (UniqueName: \"kubernetes.io/projected/e680f714-8be8-4c3c-9ba9-407d103117d7-kube-api-access-nbglq\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:03.803375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.803340    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6fbpp\" (UniqueName: \"kubernetes.io/projected/0e2914cb-1d72-48a5-a127-9f46e43566f3-kube-api-access-6fbpp\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:03.803375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.803349    2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e680f714-8be8-4c3c-9ba9-407d103117d7-extensions-socket-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:03.971716 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.971687    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e2914cb-1d72-48a5-a127-9f46e43566f3" path="/var/lib/kubelet/pods/0e2914cb-1d72-48a5-a127-9f46e43566f3/volumes"
Sep 11 13:14:03.972072 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:03.972057    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e680f714-8be8-4c3c-9ba9-407d103117d7" path="/var/lib/kubelet/pods/e680f714-8be8-4c3c-9ba9-407d103117d7/volumes"
Sep 11 13:14:04.035038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.035010    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:04.036663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.036642    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"]
Sep 11 13:14:04.037856 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:04.037836    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod146648ff_6004_4daf_b36f_697f16fd35c8.slice/crio-1aa9f4e7f3204fdee84bdb4e52e22eb56e0f4972feaf66fff3d1ad9a92eccb46 WatchSource:0}: Error finding container 1aa9f4e7f3204fdee84bdb4e52e22eb56e0f4972feaf66fff3d1ad9a92eccb46: Status 404 returned error can't find the container with id 1aa9f4e7f3204fdee84bdb4e52e22eb56e0f4972feaf66fff3d1ad9a92eccb46
Sep 11 13:14:04.038483 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:04.038466    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode36be538_4cab_48d3_af51_2a2de05b8823.slice/crio-0a4ed42f371873f4fbf0732be0a5621cf8751e43d823a62fae14b1e112266f64 WatchSource:0}: Error finding container 0a4ed42f371873f4fbf0732be0a5621cf8751e43d823a62fae14b1e112266f64: Status 404 returned error can't find the container with id 0a4ed42f371873f4fbf0732be0a5621cf8751e43d823a62fae14b1e112266f64
Sep 11 13:14:04.204490 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.204462    2573 generic.go:356] "Generic (PLEG): container finished" podID="e680f714-8be8-4c3c-9ba9-407d103117d7" containerID="3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309" exitCode=0
Sep 11 13:14:04.204668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.204513    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-t2sm8"
Sep 11 13:14:04.204668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.204564    2573 scope.go:117] "RemoveContainer" containerID="3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309"
Sep 11 13:14:04.206371 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.206341    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn" event={"ID":"146648ff-6004-4daf-b36f-697f16fd35c8","Type":"ContainerStarted","Data":"42d171654e7f9088e0b5cccd4fedf4b304d8983df09ba11ff4d4fb3a74ee9235"}
Sep 11 13:14:04.206578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.206557    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn" event={"ID":"146648ff-6004-4daf-b36f-697f16fd35c8","Type":"ContainerStarted","Data":"1aa9f4e7f3204fdee84bdb4e52e22eb56e0f4972feaf66fff3d1ad9a92eccb46"}
Sep 11 13:14:04.206755 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.206738    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:04.207892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.207868    2573 generic.go:356] "Generic (PLEG): container finished" podID="0e2914cb-1d72-48a5-a127-9f46e43566f3" containerID="d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1" exitCode=0
Sep 11 13:14:04.208002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.207912    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-pvqzg"
Sep 11 13:14:04.209787 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.209766    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" event={"ID":"e36be538-4cab-48d3-af51-2a2de05b8823","Type":"ContainerStarted","Data":"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"}
Sep 11 13:14:04.209891 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.209800    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" event={"ID":"e36be538-4cab-48d3-af51-2a2de05b8823","Type":"ContainerStarted","Data":"0a4ed42f371873f4fbf0732be0a5621cf8751e43d823a62fae14b1e112266f64"}
Sep 11 13:14:04.209891 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.209874    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:04.213764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.213743    2573 scope.go:117] "RemoveContainer" containerID="3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309"
Sep 11 13:14:04.214015 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:04.213999    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309\": container with ID starting with 3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309 not found: ID does not exist" containerID="3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309"
Sep 11 13:14:04.214062 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.214021    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309"} err="failed to get container status \"3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309\": rpc error: code = NotFound desc = could not find container \"3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309\": container with ID starting with 3cac82892134315323021aac7fd72c679ee612f018b5cc38b20f17bbf700e309 not found: ID does not exist"
Sep 11 13:14:04.214062 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.214041    2573 scope.go:117] "RemoveContainer" containerID="d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1"
Sep 11 13:14:04.220950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.220933    2573 scope.go:117] "RemoveContainer" containerID="d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1"
Sep 11 13:14:04.221174 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:04.221156    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1\": container with ID starting with d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1 not found: ID does not exist" containerID="d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1"
Sep 11 13:14:04.221218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.221187    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1"} err="failed to get container status \"d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1\": rpc error: code = NotFound desc = could not find container \"d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1\": container with ID starting with d9449315e1945dae11731194d459501a1d8ae77b7ace95731a679cf61c01aab1 not found: ID does not exist"
Sep 11 13:14:04.240862 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.240818    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn" podStartSLOduration=1.240801964 podStartE2EDuration="1.240801964s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:04.237858613 +0000 UTC m=+558.757895957" watchObservedRunningTime="2026-09-11 13:14:04.240801964 +0000 UTC m=+558.760839313"
Sep 11 13:14:04.306087 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:04.305994    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" podStartSLOduration=1.30598236 podStartE2EDuration="1.30598236s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:04.305449687 +0000 UTC m=+558.825487035" watchObservedRunningTime="2026-09-11 13:14:04.30598236 +0000 UTC m=+558.826019704"
Sep 11 13:14:15.223953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:15.223925    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:16.227267 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:16.227232    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-74495c69f8-9p4mn"
Sep 11 13:14:25.432155 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.432123    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:25.432527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.432327    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" podUID="e36be538-4cab-48d3-af51-2a2de05b8823" containerName="manager" containerID="cri-o://bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff" gracePeriod=10
Sep 11 13:14:25.673098 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.673075    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:25.764688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.764662    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbs88\" (UniqueName: \"kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88\") pod \"e36be538-4cab-48d3-af51-2a2de05b8823\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") "
Sep 11 13:14:25.764841 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.764769    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume\") pod \"e36be538-4cab-48d3-af51-2a2de05b8823\" (UID: \"e36be538-4cab-48d3-af51-2a2de05b8823\") "
Sep 11 13:14:25.765118 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.765095    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "e36be538-4cab-48d3-af51-2a2de05b8823" (UID: "e36be538-4cab-48d3-af51-2a2de05b8823"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:14:25.766656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.766639    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88" (OuterVolumeSpecName: "kube-api-access-kbs88") pod "e36be538-4cab-48d3-af51-2a2de05b8823" (UID: "e36be538-4cab-48d3-af51-2a2de05b8823"). InnerVolumeSpecName "kube-api-access-kbs88". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:25.865322 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.865298    2573 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/e36be538-4cab-48d3-af51-2a2de05b8823-extensions-socket-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:25.865322 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:25.865319    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbs88\" (UniqueName: \"kubernetes.io/projected/e36be538-4cab-48d3-af51-2a2de05b8823-kube-api-access-kbs88\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:26.319449 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.319414    2573 generic.go:356] "Generic (PLEG): container finished" podID="e36be538-4cab-48d3-af51-2a2de05b8823" containerID="bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff" exitCode=0
Sep 11 13:14:26.319622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.319456    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"
Sep 11 13:14:26.319622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.319501    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" event={"ID":"e36be538-4cab-48d3-af51-2a2de05b8823","Type":"ContainerDied","Data":"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"}
Sep 11 13:14:26.319622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.319550    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl" event={"ID":"e36be538-4cab-48d3-af51-2a2de05b8823","Type":"ContainerDied","Data":"0a4ed42f371873f4fbf0732be0a5621cf8751e43d823a62fae14b1e112266f64"}
Sep 11 13:14:26.319622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.319572    2573 scope.go:117] "RemoveContainer" containerID="bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"
Sep 11 13:14:26.327717 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.327691    2573 scope.go:117] "RemoveContainer" containerID="bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"
Sep 11 13:14:26.327978 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:26.327961    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff\": container with ID starting with bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff not found: ID does not exist" containerID="bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"
Sep 11 13:14:26.328030 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.327994    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff"} err="failed to get container status \"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff\": rpc error: code = NotFound desc = could not find container \"bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff\": container with ID starting with bdf6fbd05dd720c707a48b8e53409dbc925682fe8250842827cbcd1afcce74ff not found: ID does not exist"
Sep 11 13:14:26.341149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.341119    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:26.355932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:26.355909    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-76f8f686bf-wz7wl"]
Sep 11 13:14:27.970843 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:27.970804    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e36be538-4cab-48d3-af51-2a2de05b8823" path="/var/lib/kubelet/pods/e36be538-4cab-48d3-af51-2a2de05b8823/volumes"
Sep 11 13:14:41.838763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.838720    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"]
Sep 11 13:14:41.839184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.839149    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e36be538-4cab-48d3-af51-2a2de05b8823" containerName="manager"
Sep 11 13:14:41.839184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.839164    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e36be538-4cab-48d3-af51-2a2de05b8823" containerName="manager"
Sep 11 13:14:41.839252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.839234    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e36be538-4cab-48d3-af51-2a2de05b8823" containerName="manager"
Sep 11 13:14:41.842392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.842372    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.844917 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.844890    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-99x7m\""
Sep 11 13:14:41.845061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.844969    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
Sep 11 13:14:41.854493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.854469    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"]
Sep 11 13:14:41.877940 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.877906    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.877959    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.877985    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878050    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a837d58-46f0-4315-97bb-52317f2e9547-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878086    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btnzf\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-kube-api-access-btnzf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878115    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878132    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a837d58-46f0-4315-97bb-52317f2e9547-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878151    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.878277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.878213    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979394 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979417    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979452    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979480    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979624    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a837d58-46f0-4315-97bb-52317f2e9547-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979681    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btnzf\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-kube-api-access-btnzf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979732    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979757    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a837d58-46f0-4315-97bb-52317f2e9547-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979788    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.979840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.979794    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-credential-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.980148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.980117    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-certs\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.980148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.980135    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-data\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.980272 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.980252    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-workload-socket\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.980369 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.980349    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1a837d58-46f0-4315-97bb-52317f2e9547-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.982325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.982303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1a837d58-46f0-4315-97bb-52317f2e9547-istio-envoy\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.982455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.982438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1a837d58-46f0-4315-97bb-52317f2e9547-istio-podinfo\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.990078 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.990052    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-istio-token\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:41.990509 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:41.990491    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btnzf\" (UniqueName: \"kubernetes.io/projected/1a837d58-46f0-4315-97bb-52317f2e9547-kube-api-access-btnzf\") pod \"maas-default-gateway-openshift-default-78bb78ff78-zbkh4\" (UID: \"1a837d58-46f0-4315-97bb-52317f2e9547\") " pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:42.154936 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:42.154843    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:42.431540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:42.431515    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"]
Sep 11 13:14:42.433594 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:42.433563    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a837d58_46f0_4315_97bb_52317f2e9547.slice/crio-8cf1ae350f78c551db522891eb31afdc672d784fd5f610f3d2f150c68176b953 WatchSource:0}: Error finding container 8cf1ae350f78c551db522891eb31afdc672d784fd5f610f3d2f150c68176b953: Status 404 returned error can't find the container with id 8cf1ae350f78c551db522891eb31afdc672d784fd5f610f3d2f150c68176b953
Sep 11 13:14:43.398495 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:43.398458    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" event={"ID":"1a837d58-46f0-4315-97bb-52317f2e9547","Type":"ContainerStarted","Data":"8cf1ae350f78c551db522891eb31afdc672d784fd5f610f3d2f150c68176b953"}
Sep 11 13:14:44.679782 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:44.679728    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:14:44.680166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:44.679832    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:14:44.680166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:44.679869    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:14:45.415121 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:45.415086    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" event={"ID":"1a837d58-46f0-4315-97bb-52317f2e9547","Type":"ContainerStarted","Data":"542d4157c482c8b0ae1f6959056197d17dc57bd82198b3f4a874e1a5f59b4969"}
Sep 11 13:14:45.439891 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:45.439840    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podStartSLOduration=2.195858038 podStartE2EDuration="4.439825177s" totalImagesPullingTime="2.243967139s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:41 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:45.43877024 +0000 UTC m=+599.958807587" watchObservedRunningTime="2026-09-11 13:14:45.439825177 +0000 UTC m=+599.959862522"
Sep 11 13:14:46.155275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.155174    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:46.160067 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.160039    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:46.419188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.419100    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:46.420150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.420127    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:14:46.584127 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.584091    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:14:46.588473 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.588452    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.591687 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.591657    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
Sep 11 13:14:46.598687 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.598669    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:14:46.617251 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.617232    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.617343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.617271    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4lh5j\" (UniqueName: \"kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.674696 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.674620    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:14:46.717902 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.717874    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.718033 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.717925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4lh5j\" (UniqueName: \"kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.718516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.718495    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.726305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.726286    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4lh5j\" (UniqueName: \"kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j\") pod \"limitador-limitador-8588749977-h8cpw\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") " pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.898419 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.898384    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:14:46.910779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.910754    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:14:46.914550 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.914524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:46.927810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.927698    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:14:46.956433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:46.956405    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:14:47.020049 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.020013    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tv5hg\" (UniqueName: \"kubernetes.io/projected/a262c75f-c0f1-4cef-a93a-67091b4559ba-kube-api-access-tv5hg\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.020224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.020067    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a262c75f-c0f1-4cef-a93a-67091b4559ba-config-file\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.120450 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.120423    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tv5hg\" (UniqueName: \"kubernetes.io/projected/a262c75f-c0f1-4cef-a93a-67091b4559ba-kube-api-access-tv5hg\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.120631 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.120471    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a262c75f-c0f1-4cef-a93a-67091b4559ba-config-file\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.121149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.121122    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/a262c75f-c0f1-4cef-a93a-67091b4559ba-config-file\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.139541 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.139509    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tv5hg\" (UniqueName: \"kubernetes.io/projected/a262c75f-c0f1-4cef-a93a-67091b4559ba-kube-api-access-tv5hg\") pod \"limitador-limitador-84994f7ffc-4lxzk\" (UID: \"a262c75f-c0f1-4cef-a93a-67091b4559ba\") " pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.188233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.188208    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:14:47.189929 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:47.189906    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a1497e9_07b7_4da1_b1d5_a4a87f15cd8f.slice/crio-22f37edea023eb442aa8e95c8d9f0c785db410b746c654b4f8bd3ab3fa2527a4 WatchSource:0}: Error finding container 22f37edea023eb442aa8e95c8d9f0c785db410b746c654b4f8bd3ab3fa2527a4: Status 404 returned error can't find the container with id 22f37edea023eb442aa8e95c8d9f0c785db410b746c654b4f8bd3ab3fa2527a4
Sep 11 13:14:47.228854 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.228822    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:47.424406 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.424363    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" event={"ID":"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f","Type":"ContainerStarted","Data":"22f37edea023eb442aa8e95c8d9f0c785db410b746c654b4f8bd3ab3fa2527a4"}
Sep 11 13:14:47.516930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.516906    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:14:47.518599 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:47.518568    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda262c75f_c0f1_4cef_a93a_67091b4559ba.slice/crio-ea758d1d3168de1473806766b9d1254d7e9481a86cdfca63f4c65d118ffbd6ac WatchSource:0}: Error finding container ea758d1d3168de1473806766b9d1254d7e9481a86cdfca63f4c65d118ffbd6ac: Status 404 returned error can't find the container with id ea758d1d3168de1473806766b9d1254d7e9481a86cdfca63f4c65d118ffbd6ac
Sep 11 13:14:47.676867 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.676831    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:47.681246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.681225    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:47.685419 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.685400    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85rqp\""
Sep 11 13:14:47.700377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.700349    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:47.723618 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.723585    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dlm7\" (UniqueName: \"kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7\") pod \"authorino-7bcfd8f9cd-78hsg\" (UID: \"f0823efd-84e0-4799-a44c-278854274216\") " pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:47.824476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.824381    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dlm7\" (UniqueName: \"kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7\") pod \"authorino-7bcfd8f9cd-78hsg\" (UID: \"f0823efd-84e0-4799-a44c-278854274216\") " pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:47.835753 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.835693    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dlm7\" (UniqueName: \"kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7\") pod \"authorino-7bcfd8f9cd-78hsg\" (UID: \"f0823efd-84e0-4799-a44c-278854274216\") " pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:47.990695 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:47.990660    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:48.269756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:48.269727    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:48.360556 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:14:48.360511    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf0823efd_84e0_4799_a44c_278854274216.slice/crio-5b0c81879cc8e1b83ee8a3ba1a4b49f6bbba47f408fbf8d7ab1a4f8b42aef6ff WatchSource:0}: Error finding container 5b0c81879cc8e1b83ee8a3ba1a4b49f6bbba47f408fbf8d7ab1a4f8b42aef6ff: Status 404 returned error can't find the container with id 5b0c81879cc8e1b83ee8a3ba1a4b49f6bbba47f408fbf8d7ab1a4f8b42aef6ff
Sep 11 13:14:48.429886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:48.429855    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" event={"ID":"f0823efd-84e0-4799-a44c-278854274216","Type":"ContainerStarted","Data":"5b0c81879cc8e1b83ee8a3ba1a4b49f6bbba47f408fbf8d7ab1a4f8b42aef6ff"}
Sep 11 13:14:48.431149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:48.431126    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk" event={"ID":"a262c75f-c0f1-4cef-a93a-67091b4559ba","Type":"ContainerStarted","Data":"ea758d1d3168de1473806766b9d1254d7e9481a86cdfca63f4c65d118ffbd6ac"}
Sep 11 13:14:49.437983 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:49.437945    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk" event={"ID":"a262c75f-c0f1-4cef-a93a-67091b4559ba","Type":"ContainerStarted","Data":"1a7192dd5e2264828016d89de2880a5bf2ed9122e283b568237a1865b672dc93"}
Sep 11 13:14:49.438384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:49.438079    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:14:49.458470 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:49.458415    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk" podStartSLOduration=2.555622249 podStartE2EDuration="3.458398805s" totalImagesPullingTime="902.776556ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:49.45709779 +0000 UTC m=+603.977135136" watchObservedRunningTime="2026-09-11 13:14:49.458398805 +0000 UTC m=+603.978436149"
Sep 11 13:14:51.451014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:51.450979    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" event={"ID":"f0823efd-84e0-4799-a44c-278854274216","Type":"ContainerStarted","Data":"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"}
Sep 11 13:14:51.468447 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:51.468402    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" podStartSLOduration=2.274720381 podStartE2EDuration="4.468389187s" totalImagesPullingTime="2.193668806s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:14:51.466904117 +0000 UTC m=+605.986941484" watchObservedRunningTime="2026-09-11 13:14:51.468389187 +0000 UTC m=+605.988426531"
Sep 11 13:14:51.574304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:51.574270    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:53.460091 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:53.460052    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" podUID="f0823efd-84e0-4799-a44c-278854274216" containerName="authorino" containerID="cri-o://338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3" gracePeriod=30
Sep 11 13:14:53.695615 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:53.695589    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:53.769020 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:53.768995    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dlm7\" (UniqueName: \"kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7\") pod \"f0823efd-84e0-4799-a44c-278854274216\" (UID: \"f0823efd-84e0-4799-a44c-278854274216\") "
Sep 11 13:14:53.770945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:53.770920    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7" (OuterVolumeSpecName: "kube-api-access-8dlm7") pod "f0823efd-84e0-4799-a44c-278854274216" (UID: "f0823efd-84e0-4799-a44c-278854274216"). InnerVolumeSpecName "kube-api-access-8dlm7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:14:53.869501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:53.869471    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dlm7\" (UniqueName: \"kubernetes.io/projected/f0823efd-84e0-4799-a44c-278854274216-kube-api-access-8dlm7\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:14:54.465967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.465934    2573 generic.go:356] "Generic (PLEG): container finished" podID="f0823efd-84e0-4799-a44c-278854274216" containerID="338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3" exitCode=0
Sep 11 13:14:54.465967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.465970    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg"
Sep 11 13:14:54.466442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.465973    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" event={"ID":"f0823efd-84e0-4799-a44c-278854274216","Type":"ContainerDied","Data":"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"}
Sep 11 13:14:54.466442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.466017    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7bcfd8f9cd-78hsg" event={"ID":"f0823efd-84e0-4799-a44c-278854274216","Type":"ContainerDied","Data":"5b0c81879cc8e1b83ee8a3ba1a4b49f6bbba47f408fbf8d7ab1a4f8b42aef6ff"}
Sep 11 13:14:54.466442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.466038    2573 scope.go:117] "RemoveContainer" containerID="338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"
Sep 11 13:14:54.474171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.474155    2573 scope.go:117] "RemoveContainer" containerID="338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"
Sep 11 13:14:54.474446 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:14:54.474428    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3\": container with ID starting with 338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3 not found: ID does not exist" containerID="338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"
Sep 11 13:14:54.474496 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.474454    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3"} err="failed to get container status \"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3\": rpc error: code = NotFound desc = could not find container \"338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3\": container with ID starting with 338ede9dd78637c68fc950b9bafba715a9e0612ccc761755541415f5c8be57e3 not found: ID does not exist"
Sep 11 13:14:54.481973 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.481950    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:54.487784 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:54.487764    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7bcfd8f9cd-78hsg"]
Sep 11 13:14:55.973880 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:14:55.972188    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f0823efd-84e0-4799-a44c-278854274216" path="/var/lib/kubelet/pods/f0823efd-84e0-4799-a44c-278854274216/volumes"
Sep 11 13:15:00.446109 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:00.446082    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"
Sep 11 13:15:00.499126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:00.499096    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:15:01.499765 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:01.499725    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" event={"ID":"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f","Type":"ContainerStarted","Data":"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"}
Sep 11 13:15:01.500167 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:01.499830    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" podUID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" containerName="limitador" containerID="cri-o://de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2" gracePeriod=30
Sep 11 13:15:01.500167 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:01.499916    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:15:01.518596 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:01.518500    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" podStartSLOduration=1.449520899 podStartE2EDuration="15.518485674s" totalImagesPullingTime="14.068964775s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:14:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:01.517257577 +0000 UTC m=+616.037294920" watchObservedRunningTime="2026-09-11 13:15:01.518485674 +0000 UTC m=+616.038523019"
Sep 11 13:15:02.047511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.047483    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:15:02.133932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.133851    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file\") pod \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") "
Sep 11 13:15:02.133932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.133891    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4lh5j\" (UniqueName: \"kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j\") pod \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\" (UID: \"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f\") "
Sep 11 13:15:02.134246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.134226    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file" (OuterVolumeSpecName: "config-file") pod "3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" (UID: "3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:15:02.135954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.135934    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j" (OuterVolumeSpecName: "kube-api-access-4lh5j") pod "3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" (UID: "3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f"). InnerVolumeSpecName "kube-api-access-4lh5j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:15:02.235101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.235072    2573 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-config-file\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:15:02.235101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.235099    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4lh5j\" (UniqueName: \"kubernetes.io/projected/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f-kube-api-access-4lh5j\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:15:02.505472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.505432    2573 generic.go:356] "Generic (PLEG): container finished" podID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" containerID="de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2" exitCode=0
Sep 11 13:15:02.505926 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.505483    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw"
Sep 11 13:15:02.505926 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.505522    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" event={"ID":"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f","Type":"ContainerDied","Data":"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"}
Sep 11 13:15:02.505926 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.505589    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-8588749977-h8cpw" event={"ID":"3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f","Type":"ContainerDied","Data":"22f37edea023eb442aa8e95c8d9f0c785db410b746c654b4f8bd3ab3fa2527a4"}
Sep 11 13:15:02.505926 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.505626    2573 scope.go:117] "RemoveContainer" containerID="de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"
Sep 11 13:15:02.514140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.514120    2573 scope.go:117] "RemoveContainer" containerID="de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"
Sep 11 13:15:02.514417 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:15:02.514400    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2\": container with ID starting with de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2 not found: ID does not exist" containerID="de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"
Sep 11 13:15:02.514462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.514425    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2"} err="failed to get container status \"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2\": rpc error: code = NotFound desc = could not find container \"de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2\": container with ID starting with de8bc5ec89af6dd9d2ac8a1057add65d77ae2aa0ffd6a109b3da3e04f99234b2 not found: ID does not exist"
Sep 11 13:15:02.525913 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.525882    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:15:02.530451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:02.530427    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-8588749977-h8cpw"]
Sep 11 13:15:03.129227 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129195    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"]
Sep 11 13:15:03.129571 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129553    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f0823efd-84e0-4799-a44c-278854274216" containerName="authorino"
Sep 11 13:15:03.129650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129574    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f0823efd-84e0-4799-a44c-278854274216" containerName="authorino"
Sep 11 13:15:03.129650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129598    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" containerName="limitador"
Sep 11 13:15:03.129650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129608    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" containerName="limitador"
Sep 11 13:15:03.129831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129688    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" containerName="limitador"
Sep 11 13:15:03.129831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.129701    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f0823efd-84e0-4799-a44c-278854274216" containerName="authorino"
Sep 11 13:15:03.133876 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.133855    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.136333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.136317    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\""
Sep 11 13:15:03.136430 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.136372    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\""
Sep 11 13:15:03.136430 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.136414    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\""
Sep 11 13:15:03.136559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.136430    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-hc5jh\""
Sep 11 13:15:03.140621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.140590    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"]
Sep 11 13:15:03.242909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.242873    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8j6q\" (UniqueName: \"kubernetes.io/projected/70387693-2818-4205-9e7e-405b1e82214e-kube-api-access-b8j6q\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.242909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.242912    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70387693-2818-4205-9e7e-405b1e82214e-data\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.343342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.343314    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8j6q\" (UniqueName: \"kubernetes.io/projected/70387693-2818-4205-9e7e-405b1e82214e-kube-api-access-b8j6q\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.343477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.343347    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70387693-2818-4205-9e7e-405b1e82214e-data\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.343665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.343651    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/70387693-2818-4205-9e7e-405b1e82214e-data\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.351559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.351532    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8j6q\" (UniqueName: \"kubernetes.io/projected/70387693-2818-4205-9e7e-405b1e82214e-kube-api-access-b8j6q\") pod \"postgres-69fb4594cf-nhrlh\" (UID: \"70387693-2818-4205-9e7e-405b1e82214e\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.444278 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.444223    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:03.717744 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.717620    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"]
Sep 11 13:15:03.720160 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:15:03.720134    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod70387693_2818_4205_9e7e_405b1e82214e.slice/crio-3fdc160227d7e7c9b4a777c4567b298e31c48b2c9dae4711ea35a4743f496e93 WatchSource:0}: Error finding container 3fdc160227d7e7c9b4a777c4567b298e31c48b2c9dae4711ea35a4743f496e93: Status 404 returned error can't find the container with id 3fdc160227d7e7c9b4a777c4567b298e31c48b2c9dae4711ea35a4743f496e93
Sep 11 13:15:03.971386 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:03.971311    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f" path="/var/lib/kubelet/pods/3a1497e9-07b7-4da1-b1d5-a4a87f15cd8f/volumes"
Sep 11 13:15:04.517333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:04.517294    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh" event={"ID":"70387693-2818-4205-9e7e-405b1e82214e","Type":"ContainerStarted","Data":"3fdc160227d7e7c9b4a777c4567b298e31c48b2c9dae4711ea35a4743f496e93"}
Sep 11 13:15:06.529026 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:06.528987    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh" event={"ID":"70387693-2818-4205-9e7e-405b1e82214e","Type":"ContainerStarted","Data":"97f06cfb828dce2c9111f0674d844c96a64cb62a55af41e9935147102b14b984"}
Sep 11 13:15:06.529416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:06.529111    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:06.559113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:06.559046    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh" podStartSLOduration=1.423036686 podStartE2EDuration="3.559023111s" totalImagesPullingTime="2.135986425s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:06.557843078 +0000 UTC m=+621.077880424" watchObservedRunningTime="2026-09-11 13:15:06.559023111 +0000 UTC m=+621.079060460"
Sep 11 13:15:12.560832 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:12.560763    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-nhrlh"
Sep 11 13:15:19.688280 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.688248    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:15:19.695141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.695122    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.697189 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.697157    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\""
Sep 11 13:15:19.697354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.697235    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
Sep 11 13:15:19.697354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.697240    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-metrics-tls\""
Sep 11 13:15:19.697354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.697329    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-jhc68\""
Sep 11 13:15:19.701107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.701084    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:15:19.762047 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.762022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.762162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.762060    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.762162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.762083    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t6z4l\" (UniqueName: \"kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.863543 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.863504    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.863731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.863559    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.863731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.863587    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t6z4l\" (UniqueName: \"kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.866251 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.866218    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.866376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.866218    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:19.872737 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:19.872692    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t6z4l\" (UniqueName: \"kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l\") pod \"maas-controller-7fcc9dff55-7pzzf\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") " pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:20.008482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:20.008444    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:20.277974 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:20.277950    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:15:20.279909 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:15:20.279882    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e62a940_6685_46a0_a12d_25ba95688080.slice/crio-8a854c85fa20bd6398dc8f4993007d41a05df5a6874e67ad13b1589bf1deb89a WatchSource:0}: Error finding container 8a854c85fa20bd6398dc8f4993007d41a05df5a6874e67ad13b1589bf1deb89a: Status 404 returned error can't find the container with id 8a854c85fa20bd6398dc8f4993007d41a05df5a6874e67ad13b1589bf1deb89a
Sep 11 13:15:20.590621 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:20.590549    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" event={"ID":"7e62a940-6685-46a0-a12d-25ba95688080","Type":"ContainerStarted","Data":"8a854c85fa20bd6398dc8f4993007d41a05df5a6874e67ad13b1589bf1deb89a"}
Sep 11 13:15:23.610472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:23.610432    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" event={"ID":"7e62a940-6685-46a0-a12d-25ba95688080","Type":"ContainerStarted","Data":"39b0c2bf7507f357d8e47bb847f3f794a876686fc0af7f4c3e1a5b7b0d4c78b5"}
Sep 11 13:15:23.610881 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:23.610571    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:23.632752 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:23.632697    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" podStartSLOduration=2.257548524 podStartE2EDuration="4.632684599s" totalImagesPullingTime="2.375136075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:15:23.630475304 +0000 UTC m=+638.150512648" watchObservedRunningTime="2026-09-11 13:15:23.632684599 +0000 UTC m=+638.152721943"
Sep 11 13:15:34.618387 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:34.618359    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:15:56.169019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.168984    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:15:56.175161 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.175134    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.179007 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.178986    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\""
Sep 11 13:15:56.179127 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.178989    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\""
Sep 11 13:15:56.179425 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.179405    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-zxpw2\""
Sep 11 13:15:56.187875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.187851    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:15:56.244060 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.244018    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gtblf\" (UniqueName: \"kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.244236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.244093    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.244236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.244132    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.244236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.244159    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.345478 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.345444    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gtblf\" (UniqueName: \"kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.345653 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.345496    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.345653 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.345530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.345653 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.345547    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.345862 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:15:56.345764    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-metrics-tls: secret "maas-api-metrics-tls" not found
Sep 11 13:15:56.345862 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:15:56.345841    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs podName:b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35 nodeName:}" failed. No retries permitted until 2026-09-11 13:15:56.845818944 +0000 UTC m=+671.365856287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs") pod "maas-api-549f4476fd-x4v6g" (UID: "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35") : secret "maas-api-metrics-tls" not found
Sep 11 13:15:56.346255 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.346232    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.348291 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.348270    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.356339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.356318    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gtblf\" (UniqueName: \"kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.849767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.849734    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:56.852059 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:56.852037    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") pod \"maas-api-549f4476fd-x4v6g\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") " pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:57.085005 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.084968    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:15:57.321129 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.321102    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-plp2d"]
Sep 11 13:15:57.323311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.323292    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.331308 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.331285    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\""
Sep 11 13:15:57.334477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.334460    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-rkd72\""
Sep 11 13:15:57.354602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.354576    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.354751 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.354615    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-blxd2\" (UniqueName: \"kubernetes.io/projected/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-kube-api-access-blxd2\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.391420 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.391362    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-plp2d"]
Sep 11 13:15:57.414605 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.414584    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:15:57.417139 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:15:57.417118    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb67d81c9_44ed_42c0_bd4e_f9fed7fdaf35.slice/crio-c3218b0984ac3affa1aedf18925976bcc44a0630ee1964f42e6d98ea2f3aafe7 WatchSource:0}: Error finding container c3218b0984ac3affa1aedf18925976bcc44a0630ee1964f42e6d98ea2f3aafe7: Status 404 returned error can't find the container with id c3218b0984ac3affa1aedf18925976bcc44a0630ee1964f42e6d98ea2f3aafe7
Sep 11 13:15:57.419101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.419074    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:15:57.425165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.425145    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:15:57.455363 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.455340    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-blxd2\" (UniqueName: \"kubernetes.io/projected/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-kube-api-access-blxd2\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.455498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.455424    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.456059 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.456032    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-plugins-config-volume\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.473868 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.473841    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-blxd2\" (UniqueName: \"kubernetes.io/projected/9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4-kube-api-access-blxd2\") pod \"payload-processing-5b54b7fbb7-plp2d\" (UID: \"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4\") " pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.633136 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.633105    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:15:57.770113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.770078    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" event={"ID":"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35","Type":"ContainerStarted","Data":"c3218b0984ac3affa1aedf18925976bcc44a0630ee1964f42e6d98ea2f3aafe7"}
Sep 11 13:15:57.919073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:57.919044    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-5b54b7fbb7-plp2d"]
Sep 11 13:15:57.921472 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:15:57.921445    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ad61c6c_9b85_4cbe_bace_6d34a6a54eb4.slice/crio-bec5dd96a82f596a1d54a8d1f4c5ad60f27a398c050afe1a8696e76c8cb95643 WatchSource:0}: Error finding container bec5dd96a82f596a1d54a8d1f4c5ad60f27a398c050afe1a8696e76c8cb95643: Status 404 returned error can't find the container with id bec5dd96a82f596a1d54a8d1f4c5ad60f27a398c050afe1a8696e76c8cb95643
Sep 11 13:15:58.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:15:58.778978    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d" event={"ID":"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4","Type":"ContainerStarted","Data":"bec5dd96a82f596a1d54a8d1f4c5ad60f27a398c050afe1a8696e76c8cb95643"}
Sep 11 13:16:00.790292 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.790252    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" event={"ID":"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35","Type":"ContainerStarted","Data":"f88b447bc690cb17a5fbec399a969cc15262dd44919194ed526c12eb4425197a"}
Sep 11 13:16:00.790762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.790370    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:16:00.791951 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.791930    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d" event={"ID":"9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4","Type":"ContainerStarted","Data":"e8f6101aa1b32d6ec4ae206be6ad2d380eac380f549dddfb5a1b58d828a93e4a"}
Sep 11 13:16:00.792048 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.791975    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:16:00.821975 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.821934    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" podStartSLOduration=2.211981853 podStartE2EDuration="4.82192123s" totalImagesPullingTime="2.609939377s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:00.821145029 +0000 UTC m=+675.341182374" watchObservedRunningTime="2026-09-11 13:16:00.82192123 +0000 UTC m=+675.341958575"
Sep 11 13:16:00.854473 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:00.854414    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d" podStartSLOduration=1.680088252 podStartE2EDuration="3.854395501s" totalImagesPullingTime="2.174307249s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:15:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:16:00.848737505 +0000 UTC m=+675.368774851" watchObservedRunningTime="2026-09-11 13:16:00.854395501 +0000 UTC m=+675.374432848"
Sep 11 13:16:06.803352 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.803321    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:16:06.821030 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.821003    2573 generic.go:356] "Generic (PLEG): container finished" podID="1a837d58-46f0-4315-97bb-52317f2e9547" containerID="542d4157c482c8b0ae1f6959056197d17dc57bd82198b3f4a874e1a5f59b4969" exitCode=0
Sep 11 13:16:06.821157 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.821047    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" event={"ID":"1a837d58-46f0-4315-97bb-52317f2e9547","Type":"ContainerDied","Data":"542d4157c482c8b0ae1f6959056197d17dc57bd82198b3f4a874e1a5f59b4969"}
Sep 11 13:16:06.821326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.821314    2573 scope.go:117] "RemoveContainer" containerID="542d4157c482c8b0ae1f6959056197d17dc57bd82198b3f4a874e1a5f59b4969"
Sep 11 13:16:06.822194 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.822168    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:16:06.822281 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.822230    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:16:06.822325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:06.822286    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:16:07.828240 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:07.828206    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" event={"ID":"1a837d58-46f0-4315-97bb-52317f2e9547","Type":"ContainerStarted","Data":"2d13244155c24b0c3db5b22c8d4c6b8e911fd195983db91bd3f36722617a6993"}
Sep 11 13:16:08.155826 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:08.155645    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:16:08.156090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:08.156061    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:08.156235 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:08.156209    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:09.155521 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:09.155480    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:09.156001 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:09.155554    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:10.155630 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:10.155584    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:10.156143 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:10.155663    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:11.155384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:11.155345    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:11.155671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:11.155644    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:11.800579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:11.800546    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-5b54b7fbb7-plp2d"
Sep 11 13:16:12.155846 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:12.155761    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:12.156358 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:12.156327    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:13.156258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:13.155811    2573 patch_prober.go:28] interesting pod/maas-default-gateway-openshift-default-78bb78ff78-zbkh4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused" start-of-body=
Sep 11 13:16:13.156258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:13.155886    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4" podUID="1a837d58-46f0-4315-97bb-52317f2e9547" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.29:15021/healthz/ready\": dial tcp 10.134.0.29:15021: connect: connection refused"
Sep 11 13:16:14.159402 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:14.159374    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:16:14.159808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:14.159634    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:16:14.160200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:14.160183    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-78bb78ff78-zbkh4"
Sep 11 13:16:29.921972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:29.921930    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:16:41.460839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:16:41.460753    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:02.262614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.262577    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"]
Sep 11 13:17:02.266054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.266038    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.269602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.269571    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
Sep 11 13:17:02.269759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.269618    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-2mlv2\""
Sep 11 13:17:02.270303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.270281    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
Sep 11 13:17:02.270416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.270284    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-embedding-simulated-kserve-self-signed-certs\""
Sep 11 13:17:02.293900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.293876    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"]
Sep 11 13:17:02.373974 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.373950    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.374083 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.373978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.374083 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.374037    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.374158 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.374112    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.374158 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.374151    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dwzp\" (UniqueName: \"kubernetes.io/projected/88c50909-2cc7-473f-a191-b2acf96c01f3-kube-api-access-7dwzp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.374221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.374175    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c50909-2cc7-473f-a191-b2acf96c01f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.474953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.474925    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.474965    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.474986    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dwzp\" (UniqueName: \"kubernetes.io/projected/88c50909-2cc7-473f-a191-b2acf96c01f3-kube-api-access-7dwzp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475005    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c50909-2cc7-473f-a191-b2acf96c01f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475041    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475055    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475372    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-model-cache\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475402    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-tmp-dir\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.475432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.475413    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-home\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.477201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.477182    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/88c50909-2cc7-473f-a191-b2acf96c01f3-dshm\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.477462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.477443    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/88c50909-2cc7-473f-a191-b2acf96c01f3-tls-certs\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.489556 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.489527    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dwzp\" (UniqueName: \"kubernetes.io/projected/88c50909-2cc7-473f-a191-b2acf96c01f3-kube-api-access-7dwzp\") pod \"e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx\" (UID: \"88c50909-2cc7-473f-a191-b2acf96c01f3\") " pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.577406 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.577343    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:02.862468 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:02.862363    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"]
Sep 11 13:17:02.864972 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:17:02.864941    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88c50909_2cc7_473f_a191_b2acf96c01f3.slice/crio-23ffdf83c528337d41cd8d51ce20f1e0cc6e640e8ee0f4fe3b3df6923064584b WatchSource:0}: Error finding container 23ffdf83c528337d41cd8d51ce20f1e0cc6e640e8ee0f4fe3b3df6923064584b: Status 404 returned error can't find the container with id 23ffdf83c528337d41cd8d51ce20f1e0cc6e640e8ee0f4fe3b3df6923064584b
Sep 11 13:17:03.108436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:03.108393    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx" event={"ID":"88c50909-2cc7-473f-a191-b2acf96c01f3","Type":"ContainerStarted","Data":"23ffdf83c528337d41cd8d51ce20f1e0cc6e640e8ee0f4fe3b3df6923064584b"}
Sep 11 13:17:03.365224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:03.365193    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:04.114195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:04.114155    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx" event={"ID":"88c50909-2cc7-473f-a191-b2acf96c01f3","Type":"ContainerStarted","Data":"99a1cf9ec8a9d11c7d3d6fc9c7771be853e7ee01f6ab1fb5393d77f725a496e3"}
Sep 11 13:17:04.145494 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:04.145410    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx" podStartSLOduration=1.136055011 podStartE2EDuration="2.145394925s" totalImagesPullingTime="1.009339914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:04.144676855 +0000 UTC m=+738.664714200" watchObservedRunningTime="2026-09-11 13:17:04.145394925 +0000 UTC m=+738.665432270"
Sep 11 13:17:10.327987 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:10.327950    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:12.578313 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:12.578285    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:12.578770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:12.578588    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:12.590755 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:12.590735    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:13.167702 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:13.167669    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx"
Sep 11 13:17:22.666071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:22.666031    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:38.350148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.350108    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"]
Sep 11 13:17:38.352795 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.352776    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.357800 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.357777    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\""
Sep 11 13:17:38.376254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.376222    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"]
Sep 11 13:17:38.451579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451544    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.451579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451580    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/639bf75f-2532-4560-8a3c-5e9ec1bad159-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.451883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451616    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.451883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451697    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.451883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451781    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.451883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.451802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwrbn\" (UniqueName: \"kubernetes.io/projected/639bf75f-2532-4560-8a3c-5e9ec1bad159-kube-api-access-mwrbn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552678 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552646    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552678 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552681    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mwrbn\" (UniqueName: \"kubernetes.io/projected/639bf75f-2532-4560-8a3c-5e9ec1bad159-kube-api-access-mwrbn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552729    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552744    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/639bf75f-2532-4560-8a3c-5e9ec1bad159-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552774    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.552932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.552814    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.553210 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.553185    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.553281 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.553230    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.553281 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.553260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.554961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.554943    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/639bf75f-2532-4560-8a3c-5e9ec1bad159-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.555240 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.555224    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/639bf75f-2532-4560-8a3c-5e9ec1bad159-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.561966 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.561941    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwrbn\" (UniqueName: \"kubernetes.io/projected/639bf75f-2532-4560-8a3c-5e9ec1bad159-kube-api-access-mwrbn\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-8nvjq\" (UID: \"639bf75f-2532-4560-8a3c-5e9ec1bad159\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.663209 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.663142    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:38.937744 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:38.937700    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"]
Sep 11 13:17:38.939673 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:17:38.939649    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod639bf75f_2532_4560_8a3c_5e9ec1bad159.slice/crio-b919381b0895c53567944f01a8984720af78d61bfe3a58bce734ac5e3fe94dbf WatchSource:0}: Error finding container b919381b0895c53567944f01a8984720af78d61bfe3a58bce734ac5e3fe94dbf: Status 404 returned error can't find the container with id b919381b0895c53567944f01a8984720af78d61bfe3a58bce734ac5e3fe94dbf
Sep 11 13:17:39.281872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:39.281837    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq" event={"ID":"639bf75f-2532-4560-8a3c-5e9ec1bad159","Type":"ContainerStarted","Data":"a7bf62fd8a29adfc308fe1d5f78e5936a3f5fe16096bdb5de4a8d7eef08269ac"}
Sep 11 13:17:39.282024 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:39.281890    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq" event={"ID":"639bf75f-2532-4560-8a3c-5e9ec1bad159","Type":"ContainerStarted","Data":"b919381b0895c53567944f01a8984720af78d61bfe3a58bce734ac5e3fe94dbf"}
Sep 11 13:17:39.303181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:39.303118    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq" podStartSLOduration=1.102305943 podStartE2EDuration="1.303100432s" totalImagesPullingTime="200.794489ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:39.302514751 +0000 UTC m=+773.822552097" watchObservedRunningTime="2026-09-11 13:17:39.303100432 +0000 UTC m=+773.823137777"
Sep 11 13:17:40.229449 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:40.229416    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:44.622426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.622395    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"]
Sep 11 13:17:44.625764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.625748    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.631139 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.631110    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
Sep 11 13:17:44.638021 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.637996    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"]
Sep 11 13:17:44.707147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707116    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.707319 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707156    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b713e5e7-9257-408c-844e-946d74e00463-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.707319 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707177    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r4qb2\" (UniqueName: \"kubernetes.io/projected/b713e5e7-9257-408c-844e-946d74e00463-kube-api-access-r4qb2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.707432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.707432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707363    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.707432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.707406    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808751 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808694    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808812    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b713e5e7-9257-408c-844e-946d74e00463-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808830    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r4qb2\" (UniqueName: \"kubernetes.io/projected/b713e5e7-9257-408c-844e-946d74e00463-kube-api-access-r4qb2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808888    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.808933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.808912    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.809320 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.809287    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-home\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.809444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.809322    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-model-cache\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.809444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.809328    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-tmp-dir\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.810926 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.810898    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/b713e5e7-9257-408c-844e-946d74e00463-dshm\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.811148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.811130    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/b713e5e7-9257-408c-844e-946d74e00463-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.817250 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.817226    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r4qb2\" (UniqueName: \"kubernetes.io/projected/b713e5e7-9257-408c-844e-946d74e00463-kube-api-access-r4qb2\") pod \"premium-simulated-simulated-premium-kserve-57fc66f697-8w425\" (UID: \"b713e5e7-9257-408c-844e-946d74e00463\") " pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:44.935302 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:44.935212    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:45.209824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:45.209640    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"]
Sep 11 13:17:45.212099 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:17:45.212073    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb713e5e7_9257_408c_844e_946d74e00463.slice/crio-0e70306b38835ad2ec783d5f99c35e1b75b052cae7faab50ba0df1a9ee5a6be4 WatchSource:0}: Error finding container 0e70306b38835ad2ec783d5f99c35e1b75b052cae7faab50ba0df1a9ee5a6be4: Status 404 returned error can't find the container with id 0e70306b38835ad2ec783d5f99c35e1b75b052cae7faab50ba0df1a9ee5a6be4
Sep 11 13:17:45.313284 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:45.313246    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425" event={"ID":"b713e5e7-9257-408c-844e-946d74e00463","Type":"ContainerStarted","Data":"0e70306b38835ad2ec783d5f99c35e1b75b052cae7faab50ba0df1a9ee5a6be4"}
Sep 11 13:17:46.319561 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:46.319523    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425" event={"ID":"b713e5e7-9257-408c-844e-946d74e00463","Type":"ContainerStarted","Data":"4fe6a21e66fddaef98e0023de8432603d78046f5ce2e3cdbaefd4e6975d6115b"}
Sep 11 13:17:46.445658 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:46.445593    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425" podStartSLOduration=2.238395759 podStartE2EDuration="2.445572996s" totalImagesPullingTime="207.177237ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:17:44 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:17:46.379342213 +0000 UTC m=+780.899379558" watchObservedRunningTime="2026-09-11 13:17:46.445572996 +0000 UTC m=+780.965610342"
Sep 11 13:17:46.447345 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:46.447325    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:17:48.663685 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:48.663634    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:48.664115 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:48.663724    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:48.676547 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:48.676518    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:49.345516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:49.345488    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-8nvjq"
Sep 11 13:17:54.936330 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:54.936294    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:54.936779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:54.936342    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:54.948498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:54.948476    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:17:55.372386 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:17:55.372350    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-57fc66f697-8w425"
Sep 11 13:19:18.017351 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.017319    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6bdff694-zq5s9"]
Sep 11 13:19:18.020384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.020364    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.024573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.024550    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-85rqp\""
Sep 11 13:19:18.025202 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.025179    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\""
Sep 11 13:19:18.037878 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.037855    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bdff694-zq5s9"]
Sep 11 13:19:18.156473 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.156439    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/167057a1-2653-4dbb-be89-7696c3220d48-tls-cert\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.156612 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.156489    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rz4zn\" (UniqueName: \"kubernetes.io/projected/167057a1-2653-4dbb-be89-7696c3220d48-kube-api-access-rz4zn\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.257354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.257322    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/167057a1-2653-4dbb-be89-7696c3220d48-tls-cert\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.257491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.257372    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rz4zn\" (UniqueName: \"kubernetes.io/projected/167057a1-2653-4dbb-be89-7696c3220d48-kube-api-access-rz4zn\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.259684 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.259662    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/167057a1-2653-4dbb-be89-7696c3220d48-tls-cert\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.271010 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.270952    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rz4zn\" (UniqueName: \"kubernetes.io/projected/167057a1-2653-4dbb-be89-7696c3220d48-kube-api-access-rz4zn\") pod \"authorino-6bdff694-zq5s9\" (UID: \"167057a1-2653-4dbb-be89-7696c3220d48\") " pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.329422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.329389    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6bdff694-zq5s9"
Sep 11 13:19:18.604141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.604117    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6bdff694-zq5s9"]
Sep 11 13:19:18.606896 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:18.606867    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod167057a1_2653_4dbb_be89_7696c3220d48.slice/crio-e7298a8b86c5ff70ee1dc523d035ff2dd1b58d701f5147b1e29c8f67a904e6a3 WatchSource:0}: Error finding container e7298a8b86c5ff70ee1dc523d035ff2dd1b58d701f5147b1e29c8f67a904e6a3: Status 404 returned error can't find the container with id e7298a8b86c5ff70ee1dc523d035ff2dd1b58d701f5147b1e29c8f67a904e6a3
Sep 11 13:19:18.754544 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:18.754511    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bdff694-zq5s9" event={"ID":"167057a1-2653-4dbb-be89-7696c3220d48","Type":"ContainerStarted","Data":"e7298a8b86c5ff70ee1dc523d035ff2dd1b58d701f5147b1e29c8f67a904e6a3"}
Sep 11 13:19:20.766315 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:20.766279    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6bdff694-zq5s9" event={"ID":"167057a1-2653-4dbb-be89-7696c3220d48","Type":"ContainerStarted","Data":"368adc51a7f3d765b26e893549f34f1038d155713919702e08ecd707f6c3fbf6"}
Sep 11 13:19:20.804073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:20.804010    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-6bdff694-zq5s9" podStartSLOduration=2.624546901 podStartE2EDuration="3.80398998s" totalImagesPullingTime="1.179443079s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:17 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:20.799028007 +0000 UTC m=+875.319065351" watchObservedRunningTime="2026-09-11 13:19:20.80398998 +0000 UTC m=+875.324027325"
Sep 11 13:19:40.476015 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.475938    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7705-7443b6"]
Sep 11 13:19:40.479566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.479546    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:40.482817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.482797    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6zc4w\""
Sep 11 13:19:40.492634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.492465    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7705-7443b6"]
Sep 11 13:19:40.554027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.554000    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lb9t2\" (UniqueName: \"kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2\") pod \"test-curl-7705-7443b6\" (UID: \"bfe20806-a5bf-4889-adea-f26e2d997c32\") " pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:40.654892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.654860    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lb9t2\" (UniqueName: \"kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2\") pod \"test-curl-7705-7443b6\" (UID: \"bfe20806-a5bf-4889-adea-f26e2d997c32\") " pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:40.664242 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.664218    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lb9t2\" (UniqueName: \"kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2\") pod \"test-curl-7705-7443b6\" (UID: \"bfe20806-a5bf-4889-adea-f26e2d997c32\") " pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:40.810676 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:40.810647    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:41.081645 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:41.081621    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7705-7443b6"]
Sep 11 13:19:41.083638 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:41.083607    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbfe20806_a5bf_4889_adea_f26e2d997c32.slice/crio-483eb0e96de5c71f335489397f70d227629b764756056e81c009eed67796a32f WatchSource:0}: Error finding container 483eb0e96de5c71f335489397f70d227629b764756056e81c009eed67796a32f: Status 404 returned error can't find the container with id 483eb0e96de5c71f335489397f70d227629b764756056e81c009eed67796a32f
Sep 11 13:19:41.874587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:41.874557    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7705-7443b6" event={"ID":"bfe20806-a5bf-4889-adea-f26e2d997c32","Type":"ContainerStarted","Data":"01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee"}
Sep 11 13:19:41.874587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:41.874597    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7705-7443b6" event={"ID":"bfe20806-a5bf-4889-adea-f26e2d997c32","Type":"ContainerStarted","Data":"483eb0e96de5c71f335489397f70d227629b764756056e81c009eed67796a32f"}
Sep 11 13:19:41.891552 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:41.891506    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7705-7443b6" podStartSLOduration=1.244192734 podStartE2EDuration="1.891489749s" totalImagesPullingTime="647.297015ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:41.889214761 +0000 UTC m=+896.409252107" watchObservedRunningTime="2026-09-11 13:19:41.891489749 +0000 UTC m=+896.411527093"
Sep 11 13:19:42.311889 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.311856    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:19:42.552276 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.552245    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7705-7443b6"]
Sep 11 13:19:42.558075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.557811    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7705-7443b6"]
Sep 11 13:19:42.692967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.692933    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7705-c393da"]
Sep 11 13:19:42.693288 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.693275    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" containerName="test-curl-7705-7443b6"
Sep 11 13:19:42.693333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.693290    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" containerName="test-curl-7705-7443b6"
Sep 11 13:19:42.693368 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.693363    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" containerName="test-curl-7705-7443b6"
Sep 11 13:19:42.696300 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.696283    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:42.703036 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.703003    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7705-c393da"]
Sep 11 13:19:42.773179 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.773145    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x8cd7\" (UniqueName: \"kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7\") pod \"test-curl-7705-c393da\" (UID: \"1a8b72e0-0c5f-42a1-9f84-d5d061613be6\") " pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:42.874476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.874385    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x8cd7\" (UniqueName: \"kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7\") pod \"test-curl-7705-c393da\" (UID: \"1a8b72e0-0c5f-42a1-9f84-d5d061613be6\") " pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:42.879044 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:42.879019    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7705-7443b6" container="test-curl-7705-7443b6"
Sep 11 13:19:42.882772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:42.882744    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x8cd7\" (UniqueName: \"kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7\") pod \"test-curl-7705-c393da\" (UID: \"1a8b72e0-0c5f-42a1-9f84-d5d061613be6\") " pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:43.007795 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.007760    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:43.210534 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.210459    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:19:43.214063 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.214041    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.219516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.219494    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-a-a649dc-serving-cert\""
Sep 11 13:19:43.250180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.250158    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:19:43.277715 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.277672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbbxr\" (UniqueName: \"kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.277839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.277745    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.277839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.277803    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.277944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.277919    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.296150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.296129    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7705-c393da"]
Sep 11 13:19:43.298185 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:43.298128    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1a8b72e0_0c5f_42a1_9f84_d5d061613be6.slice/crio-450f3cab8508c2f33d490bc5f4dfe4e5f7303ebb3af08259fd9f909cd978122b WatchSource:0}: Error finding container 450f3cab8508c2f33d490bc5f4dfe4e5f7303ebb3af08259fd9f909cd978122b: Status 404 returned error can't find the container with id 450f3cab8508c2f33d490bc5f4dfe4e5f7303ebb3af08259fd9f909cd978122b
Sep 11 13:19:43.378480 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.378457    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbbxr\" (UniqueName: \"kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.378586 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.378501    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.378778 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.378756    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.378877 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.378838    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.379392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.379365    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.380896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.380877    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.381496 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.381471    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.391584 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.391556    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbbxr\" (UniqueName: \"kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr\") pod \"maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.526298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.526263    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:43.834768 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.834741    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:19:43.837312 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:43.837276    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b06d932_79c5_4415_9b7e_f5260f527922.slice/crio-43f6c84dec56cd38c696140512e22ed3cb4b3e99a7225ab87dca7994aa249689 WatchSource:0}: Error finding container 43f6c84dec56cd38c696140512e22ed3cb4b3e99a7225ab87dca7994aa249689: Status 404 returned error can't find the container with id 43f6c84dec56cd38c696140512e22ed3cb4b3e99a7225ab87dca7994aa249689
Sep 11 13:19:43.885534 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.885382    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" event={"ID":"6b06d932-79c5-4415-9b7e-f5260f527922","Type":"ContainerStarted","Data":"43f6c84dec56cd38c696140512e22ed3cb4b3e99a7225ab87dca7994aa249689"}
Sep 11 13:19:43.887538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.887510    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7705-c393da" event={"ID":"1a8b72e0-0c5f-42a1-9f84-d5d061613be6","Type":"ContainerStarted","Data":"b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043"}
Sep 11 13:19:43.887655 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.887550    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7705-c393da" event={"ID":"1a8b72e0-0c5f-42a1-9f84-d5d061613be6","Type":"ContainerStarted","Data":"450f3cab8508c2f33d490bc5f4dfe4e5f7303ebb3af08259fd9f909cd978122b"}
Sep 11 13:19:43.887655 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.887544    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7705-7443b6" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" containerName="test-curl-7705-7443b6" containerID="cri-o://01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee" gracePeriod=2
Sep 11 13:19:43.889559 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:43.889530    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:43.891381 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:43.891357    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:43.913246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:43.913189    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7705-c393da" podStartSLOduration=1.9131748640000001 podStartE2EDuration="1.913174864s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:42 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:43.91097275 +0000 UTC m=+898.431010096" watchObservedRunningTime="2026-09-11 13:19:43.913174864 +0000 UTC m=+898.433212208"
Sep 11 13:19:44.603806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:44.603772    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7705-c393da"]
Sep 11 13:19:44.606456 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:44.606420    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7705-c393da"]
Sep 11 13:19:44.893931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:44.893835    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" event={"ID":"6b06d932-79c5-4415-9b7e-f5260f527922","Type":"ContainerStarted","Data":"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"}
Sep 11 13:19:44.894349 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:44.894085    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:44.894349 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:44.894105    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7705-c393da" container="test-curl-7705-c393da"
Sep 11 13:19:44.973558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:44.973507    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" podStartSLOduration=1.575174512 podStartE2EDuration="1.973493416s" totalImagesPullingTime="398.318904ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:43 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:44.972352685 +0000 UTC m=+899.492390030" watchObservedRunningTime="2026-09-11 13:19:44.973493416 +0000 UTC m=+899.493530760"
Sep 11 13:19:45.576797 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.576755    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:19:45.577294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.577277    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" containerName="test-curl-7705-c393da"
Sep 11 13:19:45.577361 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.577298    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" containerName="test-curl-7705-c393da"
Sep 11 13:19:45.577413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.577382    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" containerName="test-curl-7705-c393da"
Sep 11 13:19:45.580828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.580807    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.583324 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.583305    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-345d76-openshift-default-dockercfg-vdbgp\""
Sep 11 13:19:45.599267 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.599198    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:19:45.700587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700557    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700592    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700610    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700694    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700741    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700799    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wd949\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700988 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700836    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700988 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700865    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.700988 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.700880    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801572    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wd949\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801625    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801660    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801681    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801790    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801844    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801873    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801915    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.801982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.801943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.802308 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.802280    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.802467 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.802436    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.802538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.802496    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.802538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.802514    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.803070 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.803045    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.804318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.804282    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.804637 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.804614    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.815183 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.815156    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wd949\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.815887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.815866    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token\") pod \"e2e-autores-345d76-openshift-default-85b7599956-fgf5f\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") " pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.892756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.892726    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-autores-345d76-openshift-default-dockercfg-vdbgp\""
Sep 11 13:19:45.899533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.899495    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7705-c393da" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" containerName="test-curl-7705-c393da" containerID="cri-o://b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043" gracePeriod=2
Sep 11 13:19:45.900369 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:45.900339    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:45.901299 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:45.901272    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-c393da\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:45.976603 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:45.976346    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-c393da\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:46.062619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.062589    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:46.064652 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:46.064623    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:46.189168 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.189141    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:19:46.191134 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:46.191104    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1acb194a_ff47_4064_b162_c943aaaca0a6.slice/crio-e953ac72e0d704f2f9dffa8e42d1ab402655b816e4d4adcdfb7778dc8483ceda WatchSource:0}: Error finding container e953ac72e0d704f2f9dffa8e42d1ab402655b816e4d4adcdfb7778dc8483ceda: Status 404 returned error can't find the container with id e953ac72e0d704f2f9dffa8e42d1ab402655b816e4d4adcdfb7778dc8483ceda
Sep 11 13:19:46.193463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.193429    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:19:46.193536 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.193497    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:19:46.193536 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.193528    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:19:46.204359 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.204337    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lb9t2\" (UniqueName: \"kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2\") pod \"bfe20806-a5bf-4889-adea-f26e2d997c32\" (UID: \"bfe20806-a5bf-4889-adea-f26e2d997c32\") "
Sep 11 13:19:46.206660 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.206634    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2" (OuterVolumeSpecName: "kube-api-access-lb9t2") pod "bfe20806-a5bf-4889-adea-f26e2d997c32" (UID: "bfe20806-a5bf-4889-adea-f26e2d997c32"). InnerVolumeSpecName "kube-api-access-lb9t2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:19:46.305700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.305671    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lb9t2\" (UniqueName: \"kubernetes.io/projected/bfe20806-a5bf-4889-adea-f26e2d997c32-kube-api-access-lb9t2\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:19:46.905465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.905434    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerStarted","Data":"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"}
Sep 11 13:19:46.905912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.905483    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerStarted","Data":"e953ac72e0d704f2f9dffa8e42d1ab402655b816e4d4adcdfb7778dc8483ceda"}
Sep 11 13:19:46.906697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.906674    2573 generic.go:356] "Generic (PLEG): container finished" podID="bfe20806-a5bf-4889-adea-f26e2d997c32" containerID="01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee" exitCode=137
Sep 11 13:19:46.906808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.906723    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:46.906808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.906751    2573 scope.go:117] "RemoveContainer" containerID="01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee"
Sep 11 13:19:46.915432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.915416    2573 scope.go:117] "RemoveContainer" containerID="01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee"
Sep 11 13:19:46.915695 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:46.915675    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee\": container with ID starting with 01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee not found: ID does not exist" containerID="01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee"
Sep 11 13:19:46.915791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.915728    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee"} err="failed to get container status \"01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee\": rpc error: code = NotFound desc = could not find container \"01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee\": container with ID starting with 01114d53f8cc7e57044f46609343ef9ab97d5810ca89e9e99ca42cc438bd11ee not found: ID does not exist"
Sep 11 13:19:46.932198 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:46.932159    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podStartSLOduration=1.932145625 podStartE2EDuration="1.932145625s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:45 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:46.931076953 +0000 UTC m=+901.451114295" watchObservedRunningTime="2026-09-11 13:19:46.932145625 +0000 UTC m=+901.452182970"
Sep 11 13:19:46.933222 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:46.933196    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:46.934881 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:46.934861    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:47.901228 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:47.901192    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:47.907504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:47.907468    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:47.914651 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:47.914622    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:47.915643 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:47.915624    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:19:47.962239 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:47.962199    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:47.964476 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:47.964441    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-7443b6\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" pod="opendatahub/test-curl-7705-7443b6"
Sep 11 13:19:47.972985 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:47.972961    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfe20806-a5bf-4889-adea-f26e2d997c32" path="/var/lib/kubelet/pods/bfe20806-a5bf-4889-adea-f26e2d997c32/volumes"
Sep 11 13:19:48.107639 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.107603    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:48.223140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.223061    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x8cd7\" (UniqueName: \"kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7\") pod \"1a8b72e0-0c5f-42a1-9f84-d5d061613be6\" (UID: \"1a8b72e0-0c5f-42a1-9f84-d5d061613be6\") "
Sep 11 13:19:48.225162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.225142    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7" (OuterVolumeSpecName: "kube-api-access-x8cd7") pod "1a8b72e0-0c5f-42a1-9f84-d5d061613be6" (UID: "1a8b72e0-0c5f-42a1-9f84-d5d061613be6"). InnerVolumeSpecName "kube-api-access-x8cd7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:19:48.323752 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.323714    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x8cd7\" (UniqueName: \"kubernetes.io/projected/1a8b72e0-0c5f-42a1-9f84-d5d061613be6-kube-api-access-x8cd7\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:19:48.920994 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.920959    2573 generic.go:356] "Generic (PLEG): container finished" podID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" containerID="b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043" exitCode=137
Sep 11 13:19:48.920994 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.920997    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:48.921507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.921076    2573 scope.go:117] "RemoveContainer" containerID="b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043"
Sep 11 13:19:48.929813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.929792    2573 scope.go:117] "RemoveContainer" containerID="b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043"
Sep 11 13:19:48.930082 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:48.930064    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043\": container with ID starting with b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043 not found: ID does not exist" containerID="b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043"
Sep 11 13:19:48.930132 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:48.930088    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043"} err="failed to get container status \"b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043\": rpc error: code = NotFound desc = could not find container \"b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043\": container with ID starting with b483f8681bf6684d7ef840aee1b4ef892c6a33f2a98a8952d439029bf8b44043 not found: ID does not exist"
Sep 11 13:19:48.936258 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:48.936236    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7705-c393da\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" pod="opendatahub/test-curl-7705-c393da"
Sep 11 13:19:49.973488 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:49.973452    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1a8b72e0-0c5f-42a1-9f84-d5d061613be6" path="/var/lib/kubelet/pods/1a8b72e0-0c5f-42a1-9f84-d5d061613be6/volumes"
Sep 11 13:19:50.694733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.694320    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:19:50.699910 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.699882    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.702366 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.702338    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-autores-345d76-serving-cert\""
Sep 11 13:19:50.718145 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.718114    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:19:50.846845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.846802    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.847042 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.846861    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.847042 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.846968    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.847172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.847057    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mp5bk\" (UniqueName: \"kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.905579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.905550    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:19:50.948116 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.948027    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.948116 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.948094    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.948338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.948167    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.948338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.948242    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mp5bk\" (UniqueName: \"kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.948982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.948951    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:50.949161 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:50.949136    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-autores-345d76-serving-cert: secret "maas-api-e2e-autores-345d76-serving-cert" not found
Sep 11 13:19:50.949291 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:19:50.949229    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls podName:06abd1ab-2b75-42e4-868d-eb23c1d88eba nodeName:}" failed. No retries permitted until 2026-09-11 13:19:51.449210619 +0000 UTC m=+905.969247947 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls") pod "maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" (UID: "06abd1ab-2b75-42e4-868d-eb23c1d88eba") : secret "maas-api-e2e-autores-345d76-serving-cert" not found
Sep 11 13:19:50.951587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:50.951562    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:51.001276 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.001239    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mp5bk\" (UniqueName: \"kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:51.454436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.454398    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:51.456790 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.456762    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") pod \"maas-api-e2e-autores-345d76-688bcfbb59-mrhfb\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") " pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:51.614467 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.614433    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:51.711674 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.711608    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:19:51.716989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.716964    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.720056 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.720028    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-autores-345d76\""
Sep 11 13:19:51.720124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.720091    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-autores-345d76-dockercfg-h9xd6\""
Sep 11 13:19:51.729336 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.729314    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:19:51.857938 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.857903    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9rzzg\" (UniqueName: \"kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.858160 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.857957    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.887928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.887902    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:19:51.889693 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:51.889669    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06abd1ab_2b75_42e4_868d_eb23c1d88eba.slice/crio-54c229902ac6407d3bed4a2d0ffbe8871c1e5c85c9199c728a78b95e68b6d6fc WatchSource:0}: Error finding container 54c229902ac6407d3bed4a2d0ffbe8871c1e5c85c9199c728a78b95e68b6d6fc: Status 404 returned error can't find the container with id 54c229902ac6407d3bed4a2d0ffbe8871c1e5c85c9199c728a78b95e68b6d6fc
Sep 11 13:19:51.948007 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.947969    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" event={"ID":"06abd1ab-2b75-42e4-868d-eb23c1d88eba","Type":"ContainerStarted","Data":"54c229902ac6407d3bed4a2d0ffbe8871c1e5c85c9199c728a78b95e68b6d6fc"}
Sep 11 13:19:51.959428 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.959402    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.959573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.959561    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9rzzg\" (UniqueName: \"kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.960198 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.960179    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:51.969264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:51.969203    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9rzzg\" (UniqueName: \"kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg\") pod \"payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") " pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:52.027700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.027673    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:52.308832 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.308807    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:19:52.311869 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:52.311840    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd139137_3a25_4e26_a74d_400d694d352c.slice/crio-a75f22cdee95316d694bcd0c3cb36d3f6275c5cd655914bbea31bcfee943eb1c WatchSource:0}: Error finding container a75f22cdee95316d694bcd0c3cb36d3f6275c5cd655914bbea31bcfee943eb1c: Status 404 returned error can't find the container with id a75f22cdee95316d694bcd0c3cb36d3f6275c5cd655914bbea31bcfee943eb1c
Sep 11 13:19:52.944510 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.944475    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:19:52.948000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.947979    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:52.954085 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.954064    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-687dcb39-dockercfg-7r9xc\""
Sep 11 13:19:52.954504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.954254    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-687dcb39\""
Sep 11 13:19:52.955220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.955197    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" event={"ID":"06abd1ab-2b75-42e4-868d-eb23c1d88eba","Type":"ContainerStarted","Data":"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"}
Sep 11 13:19:52.955343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.955291    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:52.956863 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.956840    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" event={"ID":"fd139137-3a25-4e26-a74d-400d694d352c","Type":"ContainerStarted","Data":"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"}
Sep 11 13:19:52.956965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.956883    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:19:52.956965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.956899    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" event={"ID":"fd139137-3a25-4e26-a74d-400d694d352c","Type":"ContainerStarted","Data":"a75f22cdee95316d694bcd0c3cb36d3f6275c5cd655914bbea31bcfee943eb1c"}
Sep 11 13:19:52.963237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.963215    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:19:52.993400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:52.993354    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" podStartSLOduration=2.675812649 podStartE2EDuration="2.993340874s" totalImagesPullingTime="317.528225ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:52.99167443 +0000 UTC m=+907.511711775" watchObservedRunningTime="2026-09-11 13:19:52.993340874 +0000 UTC m=+907.513378219"
Sep 11 13:19:53.010339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.010282    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" podStartSLOduration=2.010261243 podStartE2EDuration="2.010261243s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:53.007815104 +0000 UTC m=+907.527852449" watchObservedRunningTime="2026-09-11 13:19:53.010261243 +0000 UTC m=+907.530298590"
Sep 11 13:19:53.069816 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.069752    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.069816 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.069798    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-48jg5\" (UniqueName: \"kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.170670 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.170631    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.170670 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.170666    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-48jg5\" (UniqueName: \"kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.171351 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.171331    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.179497 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.179466    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-48jg5\" (UniqueName: \"kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5\") pod \"payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") " pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.270749 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.270721    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.551147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.551115    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:19:53.553528 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:19:53.553496    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod31de380a_dfc9_4410_80cd_4dfbb38d0e66.slice/crio-bd730e7914da45b202fb8e2677034abcf72a05d91dbaa84c79c401a7efa3422c WatchSource:0}: Error finding container bd730e7914da45b202fb8e2677034abcf72a05d91dbaa84c79c401a7efa3422c: Status 404 returned error can't find the container with id bd730e7914da45b202fb8e2677034abcf72a05d91dbaa84c79c401a7efa3422c
Sep 11 13:19:53.973337 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.973267    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:19:53.973337 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.973322    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" event={"ID":"31de380a-dfc9-4410-80cd-4dfbb38d0e66","Type":"ContainerStarted","Data":"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"}
Sep 11 13:19:53.973494 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.973348    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" event={"ID":"31de380a-dfc9-4410-80cd-4dfbb38d0e66","Type":"ContainerStarted","Data":"bd730e7914da45b202fb8e2677034abcf72a05d91dbaa84c79c401a7efa3422c"}
Sep 11 13:19:53.984287 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:53.984235    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" podStartSLOduration=1.9842201149999998 podStartE2EDuration="1.984220115s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:19:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:19:53.982900336 +0000 UTC m=+908.502937683" watchObservedRunningTime="2026-09-11 13:19:53.984220115 +0000 UTC m=+908.504257460"
Sep 11 13:19:58.971075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:58.971043    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:19:59.807297 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:19:59.807260    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:20:03.613922 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.613885    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:20:03.619018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.618990    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.621597 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.621575    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-24931b-serving-cert\""
Sep 11 13:20:03.636962 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.636931    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:20:03.774294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.774249    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.774466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.774305    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.774466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.774373    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.774466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.774452    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fx9n\" (UniqueName: \"kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.876003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.875904    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.876003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.875953    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fx9n\" (UniqueName: \"kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.876003 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.875992    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.876286 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.876028    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.876681 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.876658    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.878532 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.878511    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.878629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.878563    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.884931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.884908    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fx9n\" (UniqueName: \"kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n\") pod \"maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.931150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.931111    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:03.976042 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:03.976011    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:20:04.243737 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.243383    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:20:04.396224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.396191    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:20:04.400965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.400946    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.403475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.403455    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-inf-b-24931b\""
Sep 11 13:20:04.413741 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.413718    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:20:04.581432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.581411    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.581497 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.581442    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hg6mc\" (UniqueName: \"kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.682326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.682293    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.682326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.682330    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hg6mc\" (UniqueName: \"kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.683234 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.683194    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.691885 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.691857    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hg6mc\" (UniqueName: \"kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc\") pod \"payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") " pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.711695 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.711658    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:04.975652 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:04.975581    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:20:05.004905 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.004874    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:20:05.007374 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:20:05.007346    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb813a496_6e34_4765_8e43_4f03ab789041.slice/crio-63faf22e77f41e29509911918724f32031ba621089d3c7a70c2adc3fdae3003d WatchSource:0}: Error finding container 63faf22e77f41e29509911918724f32031ba621089d3c7a70c2adc3fdae3003d: Status 404 returned error can't find the container with id 63faf22e77f41e29509911918724f32031ba621089d3c7a70c2adc3fdae3003d
Sep 11 13:20:05.041769 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.040173    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" event={"ID":"b813a496-6e34-4765-8e43-4f03ab789041","Type":"ContainerStarted","Data":"63faf22e77f41e29509911918724f32031ba621089d3c7a70c2adc3fdae3003d"}
Sep 11 13:20:05.045006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.042174    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" event={"ID":"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3","Type":"ContainerStarted","Data":"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"}
Sep 11 13:20:05.045006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.042227    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" event={"ID":"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3","Type":"ContainerStarted","Data":"f00675f19448f18a135508bfabf1c0fce7abadc5b69f700cd0a9ba695274fe94"}
Sep 11 13:20:05.045006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.043020    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:05.083827 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:05.083777    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" podStartSLOduration=1.765985484 podStartE2EDuration="2.083756732s" totalImagesPullingTime="317.771248ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:05.08159834 +0000 UTC m=+919.601635686" watchObservedRunningTime="2026-09-11 13:20:05.083756732 +0000 UTC m=+919.603794077"
Sep 11 13:20:06.057607 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:06.057572    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" event={"ID":"b813a496-6e34-4765-8e43-4f03ab789041","Type":"ContainerStarted","Data":"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"}
Sep 11 13:20:06.058311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:06.058285    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:06.077672 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:06.077618    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" podStartSLOduration=2.077602238 podStartE2EDuration="2.077602238s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:06.074969744 +0000 UTC m=+920.595007085" watchObservedRunningTime="2026-09-11 13:20:06.077602238 +0000 UTC m=+920.597639600"
Sep 11 13:20:12.070025 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:12.069988    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:20:13.103872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.103834    2573 generic.go:356] "Generic (PLEG): container finished" podID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerID="8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8" exitCode=0
Sep 11 13:20:13.104293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.103907    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerDied","Data":"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"}
Sep 11 13:20:13.104293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.104275    2573 scope.go:117] "RemoveContainer" containerID="8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"
Sep 11 13:20:13.105305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.105273    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:13.105419 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.105353    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:13.105419 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:13.105400    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:14.110936 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:14.110900    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerStarted","Data":"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"}
Sep 11 13:20:14.901339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:14.901305    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:20:14.902503 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:14.902478    2573 patch_prober.go:28] interesting pod/e2e-autores-345d76-openshift-default-85b7599956-fgf5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:14.902605 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:14.902547    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 13:20:15.901209 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:15.901180    2573 patch_prober.go:28] interesting pod/e2e-autores-345d76-openshift-default-85b7599956-fgf5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:15.901569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:15.901232    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 13:20:16.901249 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:16.901212    2573 patch_prober.go:28] interesting pod/e2e-autores-345d76-openshift-default-85b7599956-fgf5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:16.901628 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:16.901267    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 13:20:17.900991 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:17.900959    2573 patch_prober.go:28] interesting pod/e2e-autores-345d76-openshift-default-85b7599956-fgf5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:17.901165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:17.901013    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 13:20:17.915185 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:17.915159    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:20:18.071681 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:18.071649    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:20:18.901600 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:18.901567    2573 patch_prober.go:28] interesting pod/e2e-autores-345d76-openshift-default-85b7599956-fgf5f container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused" start-of-body=
Sep 11 13:20:18.901783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:18.901616    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.44:15021/healthz/ready\": dial tcp 10.134.0.44:15021: connect: connection refused"
Sep 11 13:20:19.905539 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:19.905509    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:20:19.906507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:19.906483    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:20:21.699034 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:21.699001    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:20:21.699528 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:21.699234    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" podUID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" containerName="payload-pre-processing" containerID="cri-o://9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b" gracePeriod=30
Sep 11 13:20:21.974308 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:21.974282    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:20:22.051491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.051443    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume\") pod \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") "
Sep 11 13:20:22.051674 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.051513    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-48jg5\" (UniqueName: \"kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5\") pod \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\" (UID: \"31de380a-dfc9-4410-80cd-4dfbb38d0e66\") "
Sep 11 13:20:22.052162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.052132    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "31de380a-dfc9-4410-80cd-4dfbb38d0e66" (UID: "31de380a-dfc9-4410-80cd-4dfbb38d0e66"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:20:22.059183 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.059135    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5" (OuterVolumeSpecName: "kube-api-access-48jg5") pod "31de380a-dfc9-4410-80cd-4dfbb38d0e66" (UID: "31de380a-dfc9-4410-80cd-4dfbb38d0e66"). InnerVolumeSpecName "kube-api-access-48jg5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:20:22.153451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.153409    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/31de380a-dfc9-4410-80cd-4dfbb38d0e66-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.153451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.153446    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-48jg5\" (UniqueName: \"kubernetes.io/projected/31de380a-dfc9-4410-80cd-4dfbb38d0e66-kube-api-access-48jg5\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:20:22.159587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.159553    2573 generic.go:356] "Generic (PLEG): container finished" podID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" containerID="9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b" exitCode=0
Sep 11 13:20:22.159785 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.159613    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"
Sep 11 13:20:22.159785 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.159643    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" event={"ID":"31de380a-dfc9-4410-80cd-4dfbb38d0e66","Type":"ContainerDied","Data":"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"}
Sep 11 13:20:22.159785 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.159720    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44" event={"ID":"31de380a-dfc9-4410-80cd-4dfbb38d0e66","Type":"ContainerDied","Data":"bd730e7914da45b202fb8e2677034abcf72a05d91dbaa84c79c401a7efa3422c"}
Sep 11 13:20:22.159785 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.159754    2573 scope.go:117] "RemoveContainer" containerID="9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"
Sep 11 13:20:22.172260 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.172241    2573 scope.go:117] "RemoveContainer" containerID="9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"
Sep 11 13:20:22.172564 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:20:22.172538    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b\": container with ID starting with 9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b not found: ID does not exist" containerID="9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"
Sep 11 13:20:22.172633 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.172577    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b"} err="failed to get container status \"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b\": rpc error: code = NotFound desc = could not find container \"9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b\": container with ID starting with 9c20c70491fb5a19652a2b5ee5be8f2db198f96c61864fcc5bd223746c35da3b not found: ID does not exist"
Sep 11 13:20:22.185566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.185544    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:20:22.190195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:22.190169    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-ait-687dcb39-79469d4c8-t4z44"]
Sep 11 13:20:23.097961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.097927    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:20:23.098366 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.098351    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" containerName="payload-pre-processing"
Sep 11 13:20:23.098410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.098368    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" containerName="payload-pre-processing"
Sep 11 13:20:23.098451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.098442    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" containerName="payload-pre-processing"
Sep 11 13:20:23.102721 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.102681    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.108165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.108141    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-migrate-d0b8556b-gw-openshift-default-dockercfg-pb6qm\""
Sep 11 13:20:23.139860 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.139830    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:20:23.163117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163083    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163137    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163210    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163260    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163284    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163348    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163396    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163424    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.163533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.163450    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vsdp\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264096    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264170    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264198    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264216    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264234    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vsdp\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264325    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264368    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264450    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264634    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264725    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264700    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.264961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.264746    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.265226 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.265206    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.266918 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.266897    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.267087 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.267069    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.275582 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.275557    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vsdp\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.275794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.275777    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token\") pod \"e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") " pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.413627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.413532    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:23.696887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.696858    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:20:23.698891 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:20:23.698844    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb27de26b_cfaf_47bc_a834_70eaeed7f43e.slice/crio-86f3b792ba4f0f1ee3c16a52c1e1ed88ea40f74ee9e4b44e8647529136be7b09 WatchSource:0}: Error finding container 86f3b792ba4f0f1ee3c16a52c1e1ed88ea40f74ee9e4b44e8647529136be7b09: Status 404 returned error can't find the container with id 86f3b792ba4f0f1ee3c16a52c1e1ed88ea40f74ee9e4b44e8647529136be7b09
Sep 11 13:20:23.702000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.701955    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:23.702112 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.702060    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:23.702166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.702148    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:23.974393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:23.974314    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="31de380a-dfc9-4410-80cd-4dfbb38d0e66" path="/var/lib/kubelet/pods/31de380a-dfc9-4410-80cd-4dfbb38d0e66/volumes"
Sep 11 13:20:24.177141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.177097    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerStarted","Data":"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"}
Sep 11 13:20:24.177622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.177155    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerStarted","Data":"86f3b792ba4f0f1ee3c16a52c1e1ed88ea40f74ee9e4b44e8647529136be7b09"}
Sep 11 13:20:24.202601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.202548    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podStartSLOduration=1.2025331160000001 podStartE2EDuration="1.202533116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:20:23 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:20:24.199656755 +0000 UTC m=+938.719694099" watchObservedRunningTime="2026-09-11 13:20:24.202533116 +0000 UTC m=+938.722570512"
Sep 11 13:20:24.413795 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.413751    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:24.418583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.418555    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:24.806932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:24.806892    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:20:25.189419 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:25.189322    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:25.190291 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:25.190268    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:44.215035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:44.215003    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:20:46.312410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.312378    2573 generic.go:356] "Generic (PLEG): container finished" podID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerID="fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e" exitCode=0
Sep 11 13:20:46.312839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.312449    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerDied","Data":"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"}
Sep 11 13:20:46.312839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.312834    2573 scope.go:117] "RemoveContainer" containerID="fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"
Sep 11 13:20:46.313781 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.313749    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:46.313845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.313818    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:46.313892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:46.313847    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:20:47.320027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:47.319995    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerStarted","Data":"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"}
Sep 11 13:20:47.413886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:47.413850    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:47.414055 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:47.414027    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:47.414100 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:47.414080    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:48.414248 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:48.414216    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:48.414638 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:48.414267    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:49.414053 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:49.414020    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:49.414233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:49.414093    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:50.413983 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:50.413951    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:50.414341 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:50.414019    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:51.413927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:51.413886    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:51.414117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:51.413940    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:52.414545 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:52.414513    2573 patch_prober.go:28] interesting pod/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused" start-of-body=
Sep 11 13:20:52.414954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:52.414570    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.50:15021/healthz/ready\": dial tcp 10.134.0.50:15021: connect: connection refused"
Sep 11 13:20:53.418180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:53.418148    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:53.418569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:53.418403    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:53.419145 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:53.419127    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:20:57.362265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:57.362236    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:20:57.362852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:57.362417    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy" containerID="cri-o://39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3" gracePeriod=30
Sep 11 13:20:59.822135 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:20:59.822094    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:21:02.612369 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.612342    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:21:02.648491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648434    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648478    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648500    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648524    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648565    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648588    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648624    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.648688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648677    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.649011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648719    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2vsdp\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp\") pod \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\" (UID: \"b27de26b-cfaf-47bc-a834-70eaeed7f43e\") "
Sep 11 13:21:02.649011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648785    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:02.649011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.648807    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data" (OuterVolumeSpecName: "istio-data") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:02.649169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.649050    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:02.649169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.649112    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.649169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.649132    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.649169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.649148    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:02.649373 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.649267    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:02.650896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.650865    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token" (OuterVolumeSpecName: "istio-token") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:02.651027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.650871    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp" (OuterVolumeSpecName: "kube-api-access-2vsdp") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "kube-api-access-2vsdp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:02.651203 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.651174    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:21:02.651203 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.651191    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "b27de26b-cfaf-47bc-a834-70eaeed7f43e" (UID: "b27de26b-cfaf-47bc-a834-70eaeed7f43e"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:21:02.749873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749841    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.749873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749867    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.749873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749876    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.750081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749884    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/b27de26b-cfaf-47bc-a834-70eaeed7f43e-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.750081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749893    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.750081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749901    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/b27de26b-cfaf-47bc-a834-70eaeed7f43e-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:02.750081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:02.749911    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2vsdp\" (UniqueName: \"kubernetes.io/projected/b27de26b-cfaf-47bc-a834-70eaeed7f43e-kube-api-access-2vsdp\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:03.426883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.426845    2573 generic.go:356] "Generic (PLEG): container finished" podID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerID="39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3" exitCode=0
Sep 11 13:21:03.427075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.426910    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"
Sep 11 13:21:03.427075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.426931    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerDied","Data":"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"}
Sep 11 13:21:03.427075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.427031    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5" event={"ID":"b27de26b-cfaf-47bc-a834-70eaeed7f43e","Type":"ContainerDied","Data":"86f3b792ba4f0f1ee3c16a52c1e1ed88ea40f74ee9e4b44e8647529136be7b09"}
Sep 11 13:21:03.427075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.427061    2573 scope.go:117] "RemoveContainer" containerID="39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"
Sep 11 13:21:03.436312 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.436297    2573 scope.go:117] "RemoveContainer" containerID="fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"
Sep 11 13:21:03.444575 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.444554    2573 scope.go:117] "RemoveContainer" containerID="39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"
Sep 11 13:21:03.444902 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:21:03.444883    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3\": container with ID starting with 39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3 not found: ID does not exist" containerID="39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"
Sep 11 13:21:03.444967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.444909    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3"} err="failed to get container status \"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3\": rpc error: code = NotFound desc = could not find container \"39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3\": container with ID starting with 39e0a327c9e87ebb3fd334e4aab0bfcc22167ab836da9bf163aae5d8063799b3 not found: ID does not exist"
Sep 11 13:21:03.444967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.444926    2573 scope.go:117] "RemoveContainer" containerID="fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"
Sep 11 13:21:03.445215 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:21:03.445197    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e\": container with ID starting with fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e not found: ID does not exist" containerID="fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"
Sep 11 13:21:03.445264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.445223    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e"} err="failed to get container status \"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e\": rpc error: code = NotFound desc = could not find container \"fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e\": container with ID starting with fd02d1b1acb84ad7d48ef767e483f43b43c6c0b5caf9e10a9d3169fc7a6df36e not found: ID does not exist"
Sep 11 13:21:03.467232 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.467206    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:21:03.478602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.478572    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-migrate-d0b8556b-gw-openshift-default-57d84675d4-cwcq5"]
Sep 11 13:21:03.973799 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:03.973684    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" path="/var/lib/kubelet/pods/b27de26b-cfaf-47bc-a834-70eaeed7f43e/volumes"
Sep 11 13:21:04.990258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990226    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:04.990805 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990785    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.990858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990808    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.990858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990821    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.990858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990830    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.990966 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990923    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.990966 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.990932    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b27de26b-cfaf-47bc-a834-70eaeed7f43e" containerName="istio-proxy"
Sep 11 13:21:04.995657 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.995633    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:04.997945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.997925    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ait-cd28402c-dockercfg-qxmhq\""
Sep 11 13:21:04.998511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:04.998497    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ait-cd28402c\""
Sep 11 13:21:05.006614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.006588    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:05.071756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.071723    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.071907 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.071773    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9v8p\" (UniqueName: \"kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.173166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.173127    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.173299 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.173181    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9v8p\" (UniqueName: \"kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.174140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.174116    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.183489 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.183463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9v8p\" (UniqueName: \"kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p\") pod \"payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") " pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.306927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.306867    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:05.585103 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:05.585078    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:05.587267 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:21:05.587239    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6757d3cd_fa4a_4b7a_9253_45646ab792c1.slice/crio-46fec5f27bb0c9a5734f947a8d541aade1e59e5d9a7165d158bdf07c5b6b4f56 WatchSource:0}: Error finding container 46fec5f27bb0c9a5734f947a8d541aade1e59e5d9a7165d158bdf07c5b6b4f56: Status 404 returned error can't find the container with id 46fec5f27bb0c9a5734f947a8d541aade1e59e5d9a7165d158bdf07c5b6b4f56
Sep 11 13:21:06.446607 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:06.446572    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" event={"ID":"6757d3cd-fa4a-4b7a-9253-45646ab792c1","Type":"ContainerStarted","Data":"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"}
Sep 11 13:21:06.446997 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:06.446626    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" event={"ID":"6757d3cd-fa4a-4b7a-9253-45646ab792c1","Type":"ContainerStarted","Data":"46fec5f27bb0c9a5734f947a8d541aade1e59e5d9a7165d158bdf07c5b6b4f56"}
Sep 11 13:21:06.446997 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:06.446694    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:06.466383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:06.466340    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" podStartSLOduration=2.466327664 podStartE2EDuration="2.466327664s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:06.465019367 +0000 UTC m=+980.985056714" watchObservedRunningTime="2026-09-11 13:21:06.466327664 +0000 UTC m=+980.986365008"
Sep 11 13:21:17.454472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:17.454392    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:22.008352 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:22.008315    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:21:42.737029 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.736997    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:42.737583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.737251    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" podUID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" containerName="payload-processing" containerID="cri-o://86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7" gracePeriod=30
Sep 11 13:21:42.853452 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.853420    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:21:42.853798 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.853668    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" podUID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" containerName="maas-api" containerID="cri-o://545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92" gracePeriod=30
Sep 11 13:21:42.928965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.928934    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:21:42.929159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:42.929136    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" podUID="fd139137-3a25-4e26-a74d-400d694d352c" containerName="payload-pre-processing" containerID="cri-o://60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff" gracePeriod=30
Sep 11 13:21:43.023453 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.023424    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:21:43.050094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.050072    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:43.179033 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.178999    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9v8p\" (UniqueName: \"kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p\") pod \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") "
Sep 11 13:21:43.179224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.179118    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume\") pod \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\" (UID: \"6757d3cd-fa4a-4b7a-9253-45646ab792c1\") "
Sep 11 13:21:43.179671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.179637    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "6757d3cd-fa4a-4b7a-9253-45646ab792c1" (UID: "6757d3cd-fa4a-4b7a-9253-45646ab792c1"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.183869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.183064    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p" (OuterVolumeSpecName: "kube-api-access-p9v8p") pod "6757d3cd-fa4a-4b7a-9253-45646ab792c1" (UID: "6757d3cd-fa4a-4b7a-9253-45646ab792c1"). InnerVolumeSpecName "kube-api-access-p9v8p". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.205226 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.205202    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:21:43.216191 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.216135    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:21:43.280469 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.280390    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9v8p\" (UniqueName: \"kubernetes.io/projected/6757d3cd-fa4a-4b7a-9253-45646ab792c1-kube-api-access-p9v8p\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.280469 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.280413    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/6757d3cd-fa4a-4b7a-9253-45646ab792c1-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.381519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381489    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mp5bk\" (UniqueName: \"kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk\") pod \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") "
Sep 11 13:21:43.381821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381595    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca\") pod \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") "
Sep 11 13:21:43.381821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381633    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs\") pod \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") "
Sep 11 13:21:43.381821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381669    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9rzzg\" (UniqueName: \"kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg\") pod \"fd139137-3a25-4e26-a74d-400d694d352c\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") "
Sep 11 13:21:43.381821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381700    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") pod \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\" (UID: \"06abd1ab-2b75-42e4-868d-eb23c1d88eba\") "
Sep 11 13:21:43.381821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381747    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume\") pod \"fd139137-3a25-4e26-a74d-400d694d352c\" (UID: \"fd139137-3a25-4e26-a74d-400d694d352c\") "
Sep 11 13:21:43.382148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.381982    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "06abd1ab-2b75-42e4-868d-eb23c1d88eba" (UID: "06abd1ab-2b75-42e4-868d-eb23c1d88eba"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.382148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.382089    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "fd139137-3a25-4e26-a74d-400d694d352c" (UID: "fd139137-3a25-4e26-a74d-400d694d352c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:21:43.382148 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.382108    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/06abd1ab-2b75-42e4-868d-eb23c1d88eba-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.383900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.383867    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "06abd1ab-2b75-42e4-868d-eb23c1d88eba" (UID: "06abd1ab-2b75-42e4-868d-eb23c1d88eba"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:43.384041 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.384021    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg" (OuterVolumeSpecName: "kube-api-access-9rzzg") pod "fd139137-3a25-4e26-a74d-400d694d352c" (UID: "fd139137-3a25-4e26-a74d-400d694d352c"). InnerVolumeSpecName "kube-api-access-9rzzg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.384146 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.384127    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk" (OuterVolumeSpecName: "kube-api-access-mp5bk") pod "06abd1ab-2b75-42e4-868d-eb23c1d88eba" (UID: "06abd1ab-2b75-42e4-868d-eb23c1d88eba"). InnerVolumeSpecName "kube-api-access-mp5bk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:21:43.384236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.384213    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "06abd1ab-2b75-42e4-868d-eb23c1d88eba" (UID: "06abd1ab-2b75-42e4-868d-eb23c1d88eba"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:21:43.482697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.482658    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.482697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.482693    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9rzzg\" (UniqueName: \"kubernetes.io/projected/fd139137-3a25-4e26-a74d-400d694d352c-kube-api-access-9rzzg\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.482899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.482733    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/06abd1ab-2b75-42e4-868d-eb23c1d88eba-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.482899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.482747    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/fd139137-3a25-4e26-a74d-400d694d352c-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.482899 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.482761    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mp5bk\" (UniqueName: \"kubernetes.io/projected/06abd1ab-2b75-42e4-868d-eb23c1d88eba-kube-api-access-mp5bk\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:21:43.643877 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.643793    2573 generic.go:356] "Generic (PLEG): container finished" podID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" containerID="545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92" exitCode=0
Sep 11 13:21:43.643877 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.643868    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"
Sep 11 13:21:43.644088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.643880    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" event={"ID":"06abd1ab-2b75-42e4-868d-eb23c1d88eba","Type":"ContainerDied","Data":"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"}
Sep 11 13:21:43.644088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.643973    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb" event={"ID":"06abd1ab-2b75-42e4-868d-eb23c1d88eba","Type":"ContainerDied","Data":"54c229902ac6407d3bed4a2d0ffbe8871c1e5c85c9199c728a78b95e68b6d6fc"}
Sep 11 13:21:43.644088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.644005    2573 scope.go:117] "RemoveContainer" containerID="545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"
Sep 11 13:21:43.645451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.645423    2573 generic.go:356] "Generic (PLEG): container finished" podID="fd139137-3a25-4e26-a74d-400d694d352c" containerID="60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff" exitCode=0
Sep 11 13:21:43.645592 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.645472    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"
Sep 11 13:21:43.645592 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.645535    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" event={"ID":"fd139137-3a25-4e26-a74d-400d694d352c","Type":"ContainerDied","Data":"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"}
Sep 11 13:21:43.645592 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.645576    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj" event={"ID":"fd139137-3a25-4e26-a74d-400d694d352c","Type":"ContainerDied","Data":"a75f22cdee95316d694bcd0c3cb36d3f6275c5cd655914bbea31bcfee943eb1c"}
Sep 11 13:21:43.647112 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.647076    2573 generic.go:356] "Generic (PLEG): container finished" podID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" containerID="86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7" exitCode=0
Sep 11 13:21:43.647215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.647103    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" event={"ID":"6757d3cd-fa4a-4b7a-9253-45646ab792c1","Type":"ContainerDied","Data":"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"}
Sep 11 13:21:43.647215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.647143    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"
Sep 11 13:21:43.647215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.647153    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9" event={"ID":"6757d3cd-fa4a-4b7a-9253-45646ab792c1","Type":"ContainerDied","Data":"46fec5f27bb0c9a5734f947a8d541aade1e59e5d9a7165d158bdf07c5b6b4f56"}
Sep 11 13:21:43.655296 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.655276    2573 scope.go:117] "RemoveContainer" containerID="545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"
Sep 11 13:21:43.655627 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:21:43.655599    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92\": container with ID starting with 545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92 not found: ID does not exist" containerID="545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"
Sep 11 13:21:43.655774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.655635    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92"} err="failed to get container status \"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92\": rpc error: code = NotFound desc = could not find container \"545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92\": container with ID starting with 545c5661916d673b308a46ffa6855f97b8ab18d16868710031e4ead89b9bfe92 not found: ID does not exist"
Sep 11 13:21:43.655774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.655658    2573 scope.go:117] "RemoveContainer" containerID="60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"
Sep 11 13:21:43.664517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.664497    2573 scope.go:117] "RemoveContainer" containerID="60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"
Sep 11 13:21:43.664824 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:21:43.664805    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff\": container with ID starting with 60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff not found: ID does not exist" containerID="60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"
Sep 11 13:21:43.664892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.664829    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff"} err="failed to get container status \"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff\": rpc error: code = NotFound desc = could not find container \"60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff\": container with ID starting with 60677bf5b277383370b1b0c6a6ca2b7fb466184dee11d975cd51beaecf1f13ff not found: ID does not exist"
Sep 11 13:21:43.664892 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.664844    2573 scope.go:117] "RemoveContainer" containerID="86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"
Sep 11 13:21:43.675945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.675928    2573 scope.go:117] "RemoveContainer" containerID="86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"
Sep 11 13:21:43.676253 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:21:43.676234    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7\": container with ID starting with 86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7 not found: ID does not exist" containerID="86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"
Sep 11 13:21:43.676335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.676256    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7"} err="failed to get container status \"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7\": rpc error: code = NotFound desc = could not find container \"86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7\": container with ID starting with 86a34733be0c5e84b5a7c669b55842692fdeadeb2ef1dc46f4724c932a4f8bd7 not found: ID does not exist"
Sep 11 13:21:43.676651 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.676632    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:21:43.683313 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.683289    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-autores-345d76-688bcfbb59-mrhfb"]
Sep 11 13:21:43.693958 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.693937    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:21:43.699273 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.699253    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-autores-345d76-c95984c66-tgxwj"]
Sep 11 13:21:43.711189 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.711171    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:43.716462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.716437    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ait-cd28402c-686b959bd7-j4kn9"]
Sep 11 13:21:43.972996 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.972924    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" path="/var/lib/kubelet/pods/06abd1ab-2b75-42e4-868d-eb23c1d88eba/volumes"
Sep 11 13:21:43.973321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.973280    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" path="/var/lib/kubelet/pods/6757d3cd-fa4a-4b7a-9253-45646ab792c1/volumes"
Sep 11 13:21:43.973664 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:43.973646    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd139137-3a25-4e26-a74d-400d694d352c" path="/var/lib/kubelet/pods/fd139137-3a25-4e26-a74d-400d694d352c/volumes"
Sep 11 13:21:44.936475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.936442    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:21:44.937073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937040    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" containerName="maas-api"
Sep 11 13:21:44.937073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937076    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" containerName="maas-api"
Sep 11 13:21:44.937247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937100    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" containerName="payload-processing"
Sep 11 13:21:44.937247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937110    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" containerName="payload-processing"
Sep 11 13:21:44.937247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937144    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd139137-3a25-4e26-a74d-400d694d352c" containerName="payload-pre-processing"
Sep 11 13:21:44.937247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937155    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd139137-3a25-4e26-a74d-400d694d352c" containerName="payload-pre-processing"
Sep 11 13:21:44.937247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937240    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6757d3cd-fa4a-4b7a-9253-45646ab792c1" containerName="payload-processing"
Sep 11 13:21:44.937442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937255    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd139137-3a25-4e26-a74d-400d694d352c" containerName="payload-pre-processing"
Sep 11 13:21:44.937442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.937268    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="06abd1ab-2b75-42e4-868d-eb23c1d88eba" containerName="maas-api"
Sep 11 13:21:44.942190 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.942169    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:44.945301 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.945282    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-derive-adc38cd1-openshift-default-dockercfg-dlgng\""
Sep 11 13:21:44.962983 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:44.962962    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:21:45.094565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094533    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094575    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094664    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094757    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4nmk\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094787    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094808    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094843    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.095009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.094948    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.195979 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.195888    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.195979 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.195943    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.195990    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196049    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196083    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196110    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196143    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196183    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196521 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196227    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4nmk\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196521 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196394    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.196890 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196859    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.197027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.196911    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.197735 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.197692    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.197885 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.197847    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.198696 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.198675    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.199218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.199199    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.207748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.207696    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.207859 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.207839    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4nmk\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk\") pod \"e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") " pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.255794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.255764    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:45.540661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.540633    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:21:45.542809 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:21:45.542778    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f63655f_1494_41c4_bc86_015548c12bfe.slice/crio-041e9c28eea804a4b8262272b1ee3587801a66719b7c6242f388a52fddfd298b WatchSource:0}: Error finding container 041e9c28eea804a4b8262272b1ee3587801a66719b7c6242f388a52fddfd298b: Status 404 returned error can't find the container with id 041e9c28eea804a4b8262272b1ee3587801a66719b7c6242f388a52fddfd298b
Sep 11 13:21:45.545050 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.545017    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:45.545141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.545082    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:45.545141 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.545131    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:45.663340 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.663307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerStarted","Data":"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"}
Sep 11 13:21:45.663522 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.663362    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerStarted","Data":"041e9c28eea804a4b8262272b1ee3587801a66719b7c6242f388a52fddfd298b"}
Sep 11 13:21:45.694169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:45.694110    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podStartSLOduration=1.69409181 podStartE2EDuration="1.69409181s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:45.691531725 +0000 UTC m=+1020.211569068" watchObservedRunningTime="2026-09-11 13:21:45.69409181 +0000 UTC m=+1020.214129154"
Sep 11 13:21:46.255930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:46.255903    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:46.260783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:46.260754    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:46.668797 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:46.668695    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:46.669579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:46.669560    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:21:56.163099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.163057    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:21:56.163668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.163337    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy" containerID="cri-o://7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa" gracePeriod=30
Sep 11 13:21:56.940147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.940115    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:21:56.945236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.945215    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.948178 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.948155    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-explicit-1c07a9-openshift-default-dockercfg-w9bv4\""
Sep 11 13:21:56.998617 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998592    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998621    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998648    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998750    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkfch\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998775    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998799    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998859    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.998931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998894    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:56.999134 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:56.998930    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.021502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.021424    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:21:57.099650 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099613    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.099838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.099838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099733    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.099838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099783    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkfch\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.099838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099821    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100055 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.099876    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100132 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100107    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100210 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100192    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100250    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100284    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100329    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100618 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100592    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100747 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100602    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.100747 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.100661    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.102321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.102300    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.102441 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.102424    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.113865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.113841    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.118630 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.118610    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkfch\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch\") pod \"e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") " pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.256329 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.256308    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:57.578036 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.578011    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:21:57.578845 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:21:57.578821    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod864069fa_eb4a_4cd0_b210_c89a99d45727.slice/crio-81a7beb9a41e1a5eae25171b69f033dcb28c1823895f346bd7147fdd2b0061e5 WatchSource:0}: Error finding container 81a7beb9a41e1a5eae25171b69f033dcb28c1823895f346bd7147fdd2b0061e5: Status 404 returned error can't find the container with id 81a7beb9a41e1a5eae25171b69f033dcb28c1823895f346bd7147fdd2b0061e5
Sep 11 13:21:57.581000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.580965    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:57.581111 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.581062    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:57.581177 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.581107    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:21:57.732151 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.732118    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerStarted","Data":"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"}
Sep 11 13:21:57.732313 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:57.732183    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerStarted","Data":"81a7beb9a41e1a5eae25171b69f033dcb28c1823895f346bd7147fdd2b0061e5"}
Sep 11 13:21:58.257043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:58.257008    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:58.261896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:58.261872    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:58.290023 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:58.289978    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podStartSLOduration=2.289965166 podStartE2EDuration="2.289965166s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:21:57.769192452 +0000 UTC m=+1032.289229798" watchObservedRunningTime="2026-09-11 13:21:58.289965166 +0000 UTC m=+1032.810002512"
Sep 11 13:21:58.738327 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:58.738297    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:21:58.739533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:21:58.739512    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:22:01.160234 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.160187    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:22:01.444750 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.444696    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:22:01.544694 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544659    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.544884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544727    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.544884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544748    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.544884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544783    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.544884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.545125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.544959    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:01.545125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545079    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.545229 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545134    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.545229 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545154    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wd949\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.545229 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545163    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:01.545357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545238    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data\") pod \"1acb194a-ff47-4064-b162-c943aaaca0a6\" (UID: \"1acb194a-ff47-4064-b162-c943aaaca0a6\") "
Sep 11 13:22:01.545357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545309    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:01.545448 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545350    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:01.545524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545497    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data" (OuterVolumeSpecName: "istio-data") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:01.545608 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545580    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.545608 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545604    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/1acb194a-ff47-4064-b162-c943aaaca0a6-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.545734 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545616    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.545734 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545651    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.545734 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.545665    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.547221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.547194    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:22:01.547498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.547475    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:01.547583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.547509    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token" (OuterVolumeSpecName: "istio-token") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:01.547745 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.547719    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949" (OuterVolumeSpecName: "kube-api-access-wd949") pod "1acb194a-ff47-4064-b162-c943aaaca0a6" (UID: "1acb194a-ff47-4064-b162-c943aaaca0a6"). InnerVolumeSpecName "kube-api-access-wd949". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:01.646465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.646435    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.646465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.646461    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wd949\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-kube-api-access-wd949\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.646465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.646472    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.646684 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.646480    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/1acb194a-ff47-4064-b162-c943aaaca0a6-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:01.755767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.755732    2573 generic.go:356] "Generic (PLEG): container finished" podID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerID="7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa" exitCode=0
Sep 11 13:22:01.755928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.755787    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerDied","Data":"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"}
Sep 11 13:22:01.755928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.755831    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f" event={"ID":"1acb194a-ff47-4064-b162-c943aaaca0a6","Type":"ContainerDied","Data":"e953ac72e0d704f2f9dffa8e42d1ab402655b816e4d4adcdfb7778dc8483ceda"}
Sep 11 13:22:01.755928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.755839    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"
Sep 11 13:22:01.755928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.755854    2573 scope.go:117] "RemoveContainer" containerID="7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"
Sep 11 13:22:01.766568 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.766545    2573 scope.go:117] "RemoveContainer" containerID="8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"
Sep 11 13:22:01.775264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.775246    2573 scope.go:117] "RemoveContainer" containerID="7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"
Sep 11 13:22:01.775570 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:22:01.775548    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa\": container with ID starting with 7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa not found: ID does not exist" containerID="7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"
Sep 11 13:22:01.775641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.775578    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa"} err="failed to get container status \"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa\": rpc error: code = NotFound desc = could not find container \"7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa\": container with ID starting with 7777fc7a682172a3d6b439c4081549be8fbe66b8a0bf4f022b8feb228f445bfa not found: ID does not exist"
Sep 11 13:22:01.775641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.775595    2573 scope.go:117] "RemoveContainer" containerID="8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"
Sep 11 13:22:01.775918 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:22:01.775902    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8\": container with ID starting with 8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8 not found: ID does not exist" containerID="8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"
Sep 11 13:22:01.775992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.775921    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8"} err="failed to get container status \"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8\": rpc error: code = NotFound desc = could not find container \"8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8\": container with ID starting with 8ca1133b8bee56c87f945317ab44756b54d9cc546c1ce5c275f11b3263a0d8e8 not found: ID does not exist"
Sep 11 13:22:01.801759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.801727    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:22:01.813036 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.813005    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-autores-345d76-openshift-default-85b7599956-fgf5f"]
Sep 11 13:22:01.971881 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:01.971848    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" path="/var/lib/kubelet/pods/1acb194a-ff47-4064-b162-c943aaaca0a6/volumes"
Sep 11 13:22:04.781814 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.781780    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:22:04.782180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.782137    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.782180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.782148    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.782180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.782159    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.782180 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.782164    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.782309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.782231    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.786803 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.786781    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:04.790165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.790135    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-explicit-1c07a9\""
Sep 11 13:22:04.790654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.790637    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-explicit-1c07a9-dockercfg-s769g\""
Sep 11 13:22:04.800237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.800216    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:22:04.851726 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.848964    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:22:04.851726 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.849663    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="1acb194a-ff47-4064-b162-c943aaaca0a6" containerName="istio-proxy"
Sep 11 13:22:04.855727 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.853270    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:04.864931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.863997    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:22:04.887246 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.887208    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4f8l\" (UniqueName: \"kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:04.887398 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.887277    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:04.988353 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.988321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4h2qv\" (UniqueName: \"kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:04.988499 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.988371    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4f8l\" (UniqueName: \"kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:04.988499 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.988406    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:04.988591 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.988569    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:04.989206 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:04.989186    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:05.000928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.000905    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4f8l\" (UniqueName: \"kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l\") pod \"payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") " pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:05.089742 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.089644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4h2qv\" (UniqueName: \"kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.089742 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.089691    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.090331 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.090311    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.096479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.096452    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:05.100070 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.100052    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4h2qv\" (UniqueName: \"kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv\") pod \"payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") " pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.174498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.174382    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.413015 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.412989    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:22:05.414113 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:22:05.414089    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5136bfb_d3f6_4422_bb57_97fc73dc6fab.slice/crio-46b9fae3d06379f7ab6ecf6f4efd971198b459bc9d09831375386cd7716e3abd WatchSource:0}: Error finding container 46b9fae3d06379f7ab6ecf6f4efd971198b459bc9d09831375386cd7716e3abd: Status 404 returned error can't find the container with id 46b9fae3d06379f7ab6ecf6f4efd971198b459bc9d09831375386cd7716e3abd
Sep 11 13:22:05.501513 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.501417    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:22:05.503812 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:22:05.503791    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod00ba7ef9_6d55_4154_bbdf_5246a73b7219.slice/crio-cd2ee08aa0d911ebeae8e0f0af0a7c8019f453557d2eb5aa6f7ee727400a99f8 WatchSource:0}: Error finding container cd2ee08aa0d911ebeae8e0f0af0a7c8019f453557d2eb5aa6f7ee727400a99f8: Status 404 returned error can't find the container with id cd2ee08aa0d911ebeae8e0f0af0a7c8019f453557d2eb5aa6f7ee727400a99f8
Sep 11 13:22:05.782558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.782524    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" event={"ID":"00ba7ef9-6d55-4154-bbdf-5246a73b7219","Type":"ContainerStarted","Data":"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"}
Sep 11 13:22:05.783078 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.782586    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" event={"ID":"00ba7ef9-6d55-4154-bbdf-5246a73b7219","Type":"ContainerStarted","Data":"cd2ee08aa0d911ebeae8e0f0af0a7c8019f453557d2eb5aa6f7ee727400a99f8"}
Sep 11 13:22:05.783078 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.782615    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:05.784337 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.784305    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" event={"ID":"e5136bfb-d3f6-4422-bb57-97fc73dc6fab","Type":"ContainerStarted","Data":"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"}
Sep 11 13:22:05.784476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.784361    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" event={"ID":"e5136bfb-d3f6-4422-bb57-97fc73dc6fab","Type":"ContainerStarted","Data":"46b9fae3d06379f7ab6ecf6f4efd971198b459bc9d09831375386cd7716e3abd"}
Sep 11 13:22:05.784476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.784389    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:05.809016 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.808970    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" podStartSLOduration=1.8089541439999999 podStartE2EDuration="1.808954144s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:05.806902455 +0000 UTC m=+1040.326939800" watchObservedRunningTime="2026-09-11 13:22:05.808954144 +0000 UTC m=+1040.328991526"
Sep 11 13:22:05.843578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:05.843510    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" podStartSLOduration=1.843490141 podStartE2EDuration="1.843490141s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:22:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:22:05.841067016 +0000 UTC m=+1040.361104362" watchObservedRunningTime="2026-09-11 13:22:05.843490141 +0000 UTC m=+1040.363527487"
Sep 11 13:22:11.820073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.820038    2573 generic.go:356] "Generic (PLEG): container finished" podID="5f63655f-1494-41c4-bc86-015548c12bfe" containerID="459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa" exitCode=0
Sep 11 13:22:11.820498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.820110    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerDied","Data":"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"}
Sep 11 13:22:11.820498 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.820470    2573 scope.go:117] "RemoveContainer" containerID="459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"
Sep 11 13:22:11.821525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.821492    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:11.821628 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.821579    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:11.821694 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:11.821625    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:12.827682 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:12.827651    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerStarted","Data":"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"}
Sep 11 13:22:13.256728 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:13.256682    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:22:13.257925 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:13.257899    2573 patch_prober.go:28] interesting pod/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:22:13.258117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:13.258092    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:22:14.257038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:14.256997    2573 patch_prober.go:28] interesting pod/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:22:14.257508 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:14.257062    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:22:15.257233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:15.257191    2573 patch_prober.go:28] interesting pod/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:22:15.257904 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:15.257849    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:22:16.256193 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:16.256156    2573 patch_prober.go:28] interesting pod/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:22:16.256474 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:16.256448    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:22:16.668740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:16.668615    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:22:16.793907 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:16.793876    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:22:16.794074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:16.793946    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:22:17.256488 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:17.256449    2573 patch_prober.go:28] interesting pod/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused" start-of-body=
Sep 11 13:22:17.256859 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:17.256827    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.52:15021/healthz/ready\": dial tcp 10.134.0.52:15021: connect: connection refused"
Sep 11 13:22:18.202632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:18.202603    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:22:18.203069 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:18.202953    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy" containerID="cri-o://9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607" gracePeriod=30
Sep 11 13:22:21.891057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.890969    2573 generic.go:356] "Generic (PLEG): container finished" podID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerID="ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d" exitCode=0
Sep 11 13:22:21.891506 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.891054    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerDied","Data":"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"}
Sep 11 13:22:21.891506 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.891433    2573 scope.go:117] "RemoveContainer" containerID="ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"
Sep 11 13:22:21.892566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.892528    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:21.892640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.892611    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:21.892680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:21.892646    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:22:22.107019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:22.106985    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:22:22.898478 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:22.898440    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerStarted","Data":"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"}
Sep 11 13:22:23.256912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.256874    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:22:23.257560 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.257445    2573 patch_prober.go:28] interesting pod/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 11 13:22:23.257780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.257751    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 11 13:22:23.645150 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.644857    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:22:23.778599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778567    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.778599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778605    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.778828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778654    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x4nmk\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.778828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778689    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.778828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778742    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.778828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778777    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778834    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778849    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:23.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778861    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778919    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs\") pod \"5f63655f-1494-41c4-bc86-015548c12bfe\" (UID: \"5f63655f-1494-41c4-bc86-015548c12bfe\") "
Sep 11 13:22:23.779043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.778968    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data" (OuterVolumeSpecName: "istio-data") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:23.779314 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779041    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:22:23.779314 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779050    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:23.779314 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779248    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:23.779479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779330    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.779479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779348    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.779479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779358    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/5f63655f-1494-41c4-bc86-015548c12bfe-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.779479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779366    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.779479 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.779374    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.781075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.781042    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:22:23.781075 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.781056    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token" (OuterVolumeSpecName: "istio-token") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:23.781322 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.781299    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:22:23.781322 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.781302    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk" (OuterVolumeSpecName: "kube-api-access-x4nmk") pod "5f63655f-1494-41c4-bc86-015548c12bfe" (UID: "5f63655f-1494-41c4-bc86-015548c12bfe"). InnerVolumeSpecName "kube-api-access-x4nmk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:22:23.880211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.880130    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x4nmk\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-kube-api-access-x4nmk\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.880211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.880160    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/5f63655f-1494-41c4-bc86-015548c12bfe-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.880211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.880169    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/5f63655f-1494-41c4-bc86-015548c12bfe-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.880211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.880177    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/5f63655f-1494-41c4-bc86-015548c12bfe-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:22:23.905144 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.905111    2573 generic.go:356] "Generic (PLEG): container finished" podID="5f63655f-1494-41c4-bc86-015548c12bfe" containerID="9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607" exitCode=0
Sep 11 13:22:23.905501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.905167    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"
Sep 11 13:22:23.905501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.905186    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerDied","Data":"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"}
Sep 11 13:22:23.905501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.905238    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh" event={"ID":"5f63655f-1494-41c4-bc86-015548c12bfe","Type":"ContainerDied","Data":"041e9c28eea804a4b8262272b1ee3587801a66719b7c6242f388a52fddfd298b"}
Sep 11 13:22:23.905501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.905261    2573 scope.go:117] "RemoveContainer" containerID="9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"
Sep 11 13:22:23.914544 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.914526    2573 scope.go:117] "RemoveContainer" containerID="459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"
Sep 11 13:22:23.923242 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.923226    2573 scope.go:117] "RemoveContainer" containerID="9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"
Sep 11 13:22:23.923555 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:22:23.923536    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607\": container with ID starting with 9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607 not found: ID does not exist" containerID="9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"
Sep 11 13:22:23.923632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.923563    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607"} err="failed to get container status \"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607\": rpc error: code = NotFound desc = could not find container \"9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607\": container with ID starting with 9a723d16168ced11ddf692c3f86cbcbc66e678e7be63f7da079a0b1d9dd62607 not found: ID does not exist"
Sep 11 13:22:23.923632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.923579    2573 scope.go:117] "RemoveContainer" containerID="459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"
Sep 11 13:22:23.923874 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:22:23.923857    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa\": container with ID starting with 459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa not found: ID does not exist" containerID="459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"
Sep 11 13:22:23.923927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.923880    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa"} err="failed to get container status \"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa\": rpc error: code = NotFound desc = could not find container \"459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa\": container with ID starting with 459c5bc3c18d254a935f413c10c131623b6b06685227314c59a51851e3e104aa not found: ID does not exist"
Sep 11 13:22:23.929751 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.929730    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:22:23.936230 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.936189    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-derive-adc38cd1-openshift-default-58bf554bdd-9phgh"]
Sep 11 13:22:23.973205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:23.973175    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" path="/var/lib/kubelet/pods/5f63655f-1494-41c4-bc86-015548c12bfe/volumes"
Sep 11 13:22:24.257114 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:24.257078    2573 patch_prober.go:28] interesting pod/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 11 13:22:24.257410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:24.257378    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 11 13:22:25.257171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:25.257125    2573 patch_prober.go:28] interesting pod/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 11 13:22:25.257654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:25.257194    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 11 13:22:26.257747 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:26.257680    2573 patch_prober.go:28] interesting pod/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 11 13:22:26.258243 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:26.257773    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 11 13:22:27.257382 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:27.257344    2573 patch_prober.go:28] interesting pod/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused" start-of-body=
Sep 11 13:22:27.257798 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:27.257764    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.53:15021/healthz/ready\": dial tcp 10.134.0.53:15021: connect: connection refused"
Sep 11 13:22:28.260831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:28.260803    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:22:28.261205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:28.261066    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:22:28.261801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:28.261785    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:22:42.499528 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:22:42.499445    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:23:01.531573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:01.531540    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:23:10.241205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.241167    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:23:10.241745 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.241421    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" podUID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" containerName="payload-processing" containerID="cri-o://946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd" gracePeriod=30
Sep 11 13:23:10.249227 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.249199    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:23:10.249481 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.249424    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" podUID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" containerName="payload-pre-processing" containerID="cri-o://916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134" gracePeriod=30
Sep 11 13:23:10.497917 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.497855    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:23:10.542374 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.542351    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:23:10.592014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.591986    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4h2qv\" (UniqueName: \"kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv\") pod \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") "
Sep 11 13:23:10.592185 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.592021    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume\") pod \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\" (UID: \"00ba7ef9-6d55-4154-bbdf-5246a73b7219\") "
Sep 11 13:23:10.592393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.592371    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "00ba7ef9-6d55-4154-bbdf-5246a73b7219" (UID: "00ba7ef9-6d55-4154-bbdf-5246a73b7219"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:10.594041 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.594015    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv" (OuterVolumeSpecName: "kube-api-access-4h2qv") pod "00ba7ef9-6d55-4154-bbdf-5246a73b7219" (UID: "00ba7ef9-6d55-4154-bbdf-5246a73b7219"). InnerVolumeSpecName "kube-api-access-4h2qv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:10.693364 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.693340    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume\") pod \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") "
Sep 11 13:23:10.693493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.693404    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4f8l\" (UniqueName: \"kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l\") pod \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\" (UID: \"e5136bfb-d3f6-4422-bb57-97fc73dc6fab\") "
Sep 11 13:23:10.693614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.693603    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4h2qv\" (UniqueName: \"kubernetes.io/projected/00ba7ef9-6d55-4154-bbdf-5246a73b7219-kube-api-access-4h2qv\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.693655 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.693616    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/00ba7ef9-6d55-4154-bbdf-5246a73b7219-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.693693 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.693674    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "e5136bfb-d3f6-4422-bb57-97fc73dc6fab" (UID: "e5136bfb-d3f6-4422-bb57-97fc73dc6fab"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:10.695301 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.695281    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l" (OuterVolumeSpecName: "kube-api-access-b4f8l") pod "e5136bfb-d3f6-4422-bb57-97fc73dc6fab" (UID: "e5136bfb-d3f6-4422-bb57-97fc73dc6fab"). InnerVolumeSpecName "kube-api-access-b4f8l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:10.794801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.794731    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:10.794801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:10.794755    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4f8l\" (UniqueName: \"kubernetes.io/projected/e5136bfb-d3f6-4422-bb57-97fc73dc6fab-kube-api-access-b4f8l\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:11.150205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.150119    2573 generic.go:356] "Generic (PLEG): container finished" podID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" containerID="946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd" exitCode=0
Sep 11 13:23:11.150205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.150168    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"
Sep 11 13:23:11.150400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.150206    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" event={"ID":"00ba7ef9-6d55-4154-bbdf-5246a73b7219","Type":"ContainerDied","Data":"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"}
Sep 11 13:23:11.150400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.150260    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87" event={"ID":"00ba7ef9-6d55-4154-bbdf-5246a73b7219","Type":"ContainerDied","Data":"cd2ee08aa0d911ebeae8e0f0af0a7c8019f453557d2eb5aa6f7ee727400a99f8"}
Sep 11 13:23:11.150400 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.150288    2573 scope.go:117] "RemoveContainer" containerID="946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"
Sep 11 13:23:11.151626 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.151604    2573 generic.go:356] "Generic (PLEG): container finished" podID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" containerID="916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134" exitCode=0
Sep 11 13:23:11.151765 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.151646    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"
Sep 11 13:23:11.151765 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.151646    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" event={"ID":"e5136bfb-d3f6-4422-bb57-97fc73dc6fab","Type":"ContainerDied","Data":"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"}
Sep 11 13:23:11.151885 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.151791    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp" event={"ID":"e5136bfb-d3f6-4422-bb57-97fc73dc6fab","Type":"ContainerDied","Data":"46b9fae3d06379f7ab6ecf6f4efd971198b459bc9d09831375386cd7716e3abd"}
Sep 11 13:23:11.161258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.161239    2573 scope.go:117] "RemoveContainer" containerID="946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"
Sep 11 13:23:11.161535 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:23:11.161515    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd\": container with ID starting with 946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd not found: ID does not exist" containerID="946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"
Sep 11 13:23:11.161611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.161546    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd"} err="failed to get container status \"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd\": rpc error: code = NotFound desc = could not find container \"946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd\": container with ID starting with 946f8a7f3f5c74c5ab89bda7a66dafed6ffa87351c3b6536a71d75f4eb57f0dd not found: ID does not exist"
Sep 11 13:23:11.161611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.161570    2573 scope.go:117] "RemoveContainer" containerID="916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"
Sep 11 13:23:11.169189 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.169168    2573 scope.go:117] "RemoveContainer" containerID="916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"
Sep 11 13:23:11.169450 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:23:11.169433    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134\": container with ID starting with 916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134 not found: ID does not exist" containerID="916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"
Sep 11 13:23:11.169492 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.169457    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134"} err="failed to get container status \"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134\": rpc error: code = NotFound desc = could not find container \"916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134\": container with ID starting with 916185e2a7d47bccccabebe1a62b8cac49364670d1370270703cc7c6755f0134 not found: ID does not exist"
Sep 11 13:23:11.176416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.176396    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:23:11.182998 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.182978    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-explicit-1c07a9-787bd45565-8qz87"]
Sep 11 13:23:11.215824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.215806    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:23:11.221211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.221191    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-explicit-1c07a9-5758668d54-fzwlp"]
Sep 11 13:23:11.971763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.971730    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" path="/var/lib/kubelet/pods/00ba7ef9-6d55-4154-bbdf-5246a73b7219/volumes"
Sep 11 13:23:11.972120 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:11.972103    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" path="/var/lib/kubelet/pods/e5136bfb-d3f6-4422-bb57-97fc73dc6fab/volumes"
Sep 11 13:23:16.804014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:16.803982    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:23:18.721626 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:18.721595    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:23:18.722128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:18.721804    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy" containerID="cri-o://737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8" gracePeriod=30
Sep 11 13:23:19.466585 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.466547    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:19.466989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.466973    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" containerName="payload-processing"
Sep 11 13:23:19.467051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.466990    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" containerName="payload-processing"
Sep 11 13:23:19.467051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467014    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:23:19.467051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467020    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:23:19.467051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467033    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" containerName="payload-pre-processing"
Sep 11 13:23:19.467051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467039    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" containerName="payload-pre-processing"
Sep 11 13:23:19.467214 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467094    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="e5136bfb-d3f6-4422-bb57-97fc73dc6fab" containerName="payload-pre-processing"
Sep 11 13:23:19.467214 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467100    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:23:19.467214 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.467107    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="00ba7ef9-6d55-4154-bbdf-5246a73b7219" containerName="payload-processing"
Sep 11 13:23:19.470245 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.470229    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.472993 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.472971    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-noten-4b7627-openshift-default-dockercfg-89z4c\""
Sep 11 13:23:19.486269 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.486245    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:19.567069 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567082    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567148    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567193    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567355 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567254    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jvqc\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567355 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567431 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567353    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567474 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567429    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.567516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.567481    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668219    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668312    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668334    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668357    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668378    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jvqc\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668426 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668419    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668460    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668664    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668804    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.668852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668842    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.669074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.668983    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.669136 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.669082    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.670821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.670799    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.671100 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.671081    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.694622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.694603    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.696856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.696831    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jvqc\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc\") pod \"e2e-noten-4b7627-openshift-default-855854f996-24nhs\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") " pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:19.780840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:19.780817    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:20.088314 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.088285    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:20.088505 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:23:20.088486    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3835415e_46f0_4c44_8f7b_8d58ff4eadae.slice/crio-5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912 WatchSource:0}: Error finding container 5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912: Status 404 returned error can't find the container with id 5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912
Sep 11 13:23:20.090676 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.090643    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:23:20.090794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.090725    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:23:20.090794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.090756    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:23:20.201187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.201155    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" event={"ID":"3835415e-46f0-4c44-8f7b-8d58ff4eadae","Type":"ContainerStarted","Data":"0459501966aa89442333e0381f42f3cfd05bf80e733b6e344de8a81236ffe898"}
Sep 11 13:23:20.201338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.201211    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" event={"ID":"3835415e-46f0-4c44-8f7b-8d58ff4eadae","Type":"ContainerStarted","Data":"5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912"}
Sep 11 13:23:20.236195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.236137    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" podStartSLOduration=1.236116927 podStartE2EDuration="1.236116927s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:23:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:23:20.234589026 +0000 UTC m=+1114.754626371" watchObservedRunningTime="2026-09-11 13:23:20.236116927 +0000 UTC m=+1114.756154273"
Sep 11 13:23:20.781656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:20.781626    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:21.786374 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:21.786342    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:22.211551 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:22.211461    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:22.212280 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:22.212257    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:23.969285 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:23.969260    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:23:24.112795 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112770    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.112955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112803    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.112955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112832    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkfch\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.112955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112869    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.112955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112893    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.112955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.112930    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.113215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113007    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.113215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113031    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.113215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113074    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token\") pod \"864069fa-eb4a-4cd0-b210-c89a99d45727\" (UID: \"864069fa-eb4a-4cd0-b210-c89a99d45727\") "
Sep 11 13:23:24.113215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113076    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:24.113415 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113267    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data" (OuterVolumeSpecName: "istio-data") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:24.113415 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113318    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:24.113574 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113458    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.113574 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113477    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.113574 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113491    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.113756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:24.113756 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.113563    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:24.115169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.115145    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:23:24.115169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.115158    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch" (OuterVolumeSpecName: "kube-api-access-hkfch") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "kube-api-access-hkfch". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:24.115303 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.115256    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token" (OuterVolumeSpecName: "istio-token") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:24.115348 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.115338    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "864069fa-eb4a-4cd0-b210-c89a99d45727" (UID: "864069fa-eb4a-4cd0-b210-c89a99d45727"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:24.214056 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214031    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.214056 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214052    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkfch\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-kube-api-access-hkfch\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.214197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214063    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.214197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214074    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.214197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214081    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/864069fa-eb4a-4cd0-b210-c89a99d45727-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.214197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.214089    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/864069fa-eb4a-4cd0-b210-c89a99d45727-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:24.227264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.227234    2573 generic.go:356] "Generic (PLEG): container finished" podID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerID="737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8" exitCode=0
Sep 11 13:23:24.227375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.227298    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"
Sep 11 13:23:24.227375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.227321    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerDied","Data":"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"}
Sep 11 13:23:24.227453 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.227407    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls" event={"ID":"864069fa-eb4a-4cd0-b210-c89a99d45727","Type":"ContainerDied","Data":"81a7beb9a41e1a5eae25171b69f033dcb28c1823895f346bd7147fdd2b0061e5"}
Sep 11 13:23:24.227453 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.227438    2573 scope.go:117] "RemoveContainer" containerID="737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"
Sep 11 13:23:24.237300 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.237284    2573 scope.go:117] "RemoveContainer" containerID="ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"
Sep 11 13:23:24.244810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.244791    2573 scope.go:117] "RemoveContainer" containerID="737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"
Sep 11 13:23:24.245053 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:23:24.245035    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8\": container with ID starting with 737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8 not found: ID does not exist" containerID="737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"
Sep 11 13:23:24.245113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.245059    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8"} err="failed to get container status \"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8\": rpc error: code = NotFound desc = could not find container \"737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8\": container with ID starting with 737335a0af85919d924169700c82e195c92a77bf52728e29b3c6bdbe927a25c8 not found: ID does not exist"
Sep 11 13:23:24.245113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.245073    2573 scope.go:117] "RemoveContainer" containerID="ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"
Sep 11 13:23:24.245316 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:23:24.245300    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d\": container with ID starting with ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d not found: ID does not exist" containerID="ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"
Sep 11 13:23:24.245354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.245321    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d"} err="failed to get container status \"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d\": rpc error: code = NotFound desc = could not find container \"ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d\": container with ID starting with ae40937aa103685f8c29af6199fb81c5b3ea91057b26176205b230e58ae6aa7d not found: ID does not exist"
Sep 11 13:23:24.262077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.262051    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:23:24.275573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:24.275553    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-explicit-1c07a9-openshift-default-6fd8945496-kvtls"]
Sep 11 13:23:25.972800 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:25.972767    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" path="/var/lib/kubelet/pods/864069fa-eb4a-4cd0-b210-c89a99d45727/volumes"
Sep 11 13:23:34.837032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:34.836998    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:23:36.071375 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:36.071340    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:36.071840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:36.071569    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerName="istio-proxy" containerID="cri-o://0459501966aa89442333e0381f42f3cfd05bf80e733b6e344de8a81236ffe898" gracePeriod=30
Sep 11 13:23:37.214483 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:37.214442    2573 patch_prober.go:28] interesting pod/e2e-noten-4b7627-openshift-default-855854f996-24nhs container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=upstream connect error or disconnect/reset before headers. reset reason: remote connection failure, transport failure reason: delayed connect error: Connection refused
Sep 11 13:23:37.214916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:37.214521    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:23:41.314884 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.314856    2573 generic.go:356] "Generic (PLEG): container finished" podID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerID="0459501966aa89442333e0381f42f3cfd05bf80e733b6e344de8a81236ffe898" exitCode=0
Sep 11 13:23:41.315183 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.314913    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" event={"ID":"3835415e-46f0-4c44-8f7b-8d58ff4eadae","Type":"ContainerDied","Data":"0459501966aa89442333e0381f42f3cfd05bf80e733b6e344de8a81236ffe898"}
Sep 11 13:23:41.315183 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.314951    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs" event={"ID":"3835415e-46f0-4c44-8f7b-8d58ff4eadae","Type":"ContainerDied","Data":"5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912"}
Sep 11 13:23:41.315183 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.314966    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5b8340b19bfbefa7d362ca5c2aa1eaa7dd182bc06409545508528788bca50912"
Sep 11 13:23:41.327356 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.327337    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:41.362768 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362690    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.362883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362778    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.362883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362816    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.362883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362844    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.362883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362869    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.363092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362927    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jvqc\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.363092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362953    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.363092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.362977    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.363092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363002    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs\") pod \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\" (UID: \"3835415e-46f0-4c44-8f7b-8d58ff4eadae\") "
Sep 11 13:23:41.363092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363060    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data" (OuterVolumeSpecName: "istio-data") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:41.363343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363084    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:41.363343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363103    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:23:41.363343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363286    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:41.363343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363294    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:41.363511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363342    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.363511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363367    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.363511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.363384    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.365185 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.365140    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:23:41.365311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.365202    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:23:41.365311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.365276    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc" (OuterVolumeSpecName: "kube-api-access-5jvqc") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "kube-api-access-5jvqc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:41.365429 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.365404    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token" (OuterVolumeSpecName: "istio-token") pod "3835415e-46f0-4c44-8f7b-8d58ff4eadae" (UID: "3835415e-46f0-4c44-8f7b-8d58ff4eadae"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:23:41.464700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464669    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jvqc\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-kube-api-access-5jvqc\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.464700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464695    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.464700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464720    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.464930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464733    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3835415e-46f0-4c44-8f7b-8d58ff4eadae-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.464930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464745    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:41.464930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:41.464757    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3835415e-46f0-4c44-8f7b-8d58ff4eadae-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:23:42.320390 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:42.320358    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"
Sep 11 13:23:42.345620 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:42.345587    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:42.348600 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:42.348577    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-noten-4b7627-openshift-default-855854f996-24nhs"]
Sep 11 13:23:43.971931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:43.971904    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" path="/var/lib/kubelet/pods/3835415e-46f0-4c44-8f7b-8d58ff4eadae/volumes"
Sep 11 13:23:48.625770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:23:48.625736    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:02.141128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:02.141082    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:04.067189 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.067155    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:24:04.067694 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.067388    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" podUID="6b06d932-79c5-4415-9b7e-f5260f527922" containerName="maas-api" containerID="cri-o://16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47" gracePeriod=30
Sep 11 13:24:04.367749 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.367434    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:24:04.450593 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.450554    2573 generic.go:356] "Generic (PLEG): container finished" podID="6b06d932-79c5-4415-9b7e-f5260f527922" containerID="16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47" exitCode=0
Sep 11 13:24:04.450808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.450596    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"
Sep 11 13:24:04.450808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.450624    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" event={"ID":"6b06d932-79c5-4415-9b7e-f5260f527922","Type":"ContainerDied","Data":"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"}
Sep 11 13:24:04.450808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.450681    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s" event={"ID":"6b06d932-79c5-4415-9b7e-f5260f527922","Type":"ContainerDied","Data":"43f6c84dec56cd38c696140512e22ed3cb4b3e99a7225ab87dca7994aa249689"}
Sep 11 13:24:04.450808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.450719    2573 scope.go:117] "RemoveContainer" containerID="16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"
Sep 11 13:24:04.459645 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.459626    2573 scope.go:117] "RemoveContainer" containerID="16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"
Sep 11 13:24:04.460065 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:24:04.460006    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47\": container with ID starting with 16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47 not found: ID does not exist" containerID="16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"
Sep 11 13:24:04.460201 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.460057    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47"} err="failed to get container status \"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47\": rpc error: code = NotFound desc = could not find container \"16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47\": container with ID starting with 16a2e339969be5f4024f14a988270b329d6215c7ba7adaf757efd878c96e3e47 not found: ID does not exist"
Sep 11 13:24:04.465777 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.465750    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs\") pod \"6b06d932-79c5-4415-9b7e-f5260f527922\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") "
Sep 11 13:24:04.465918 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.465813    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls\") pod \"6b06d932-79c5-4415-9b7e-f5260f527922\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") "
Sep 11 13:24:04.465918 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.465850    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wbbxr\" (UniqueName: \"kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr\") pod \"6b06d932-79c5-4415-9b7e-f5260f527922\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") "
Sep 11 13:24:04.465918 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.465889    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca\") pod \"6b06d932-79c5-4415-9b7e-f5260f527922\" (UID: \"6b06d932-79c5-4415-9b7e-f5260f527922\") "
Sep 11 13:24:04.466552 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.466524    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6b06d932-79c5-4415-9b7e-f5260f527922" (UID: "6b06d932-79c5-4415-9b7e-f5260f527922"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:04.468442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.468417    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "6b06d932-79c5-4415-9b7e-f5260f527922" (UID: "6b06d932-79c5-4415-9b7e-f5260f527922"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:04.468578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.468484    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6b06d932-79c5-4415-9b7e-f5260f527922" (UID: "6b06d932-79c5-4415-9b7e-f5260f527922"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:04.468578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.468544    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr" (OuterVolumeSpecName: "kube-api-access-wbbxr") pod "6b06d932-79c5-4415-9b7e-f5260f527922" (UID: "6b06d932-79c5-4415-9b7e-f5260f527922"). InnerVolumeSpecName "kube-api-access-wbbxr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:04.567275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.567242    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.567275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.567271    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6b06d932-79c5-4415-9b7e-f5260f527922-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.567275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.567280    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wbbxr\" (UniqueName: \"kubernetes.io/projected/6b06d932-79c5-4415-9b7e-f5260f527922-kube-api-access-wbbxr\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.567505 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.567293    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6b06d932-79c5-4415-9b7e-f5260f527922-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:04.794522 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.794452    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:24:04.811297 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:04.811266    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-a-a649dc-cd95f779f-fvl6s"]
Sep 11 13:24:05.972503 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:05.972471    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6b06d932-79c5-4415-9b7e-f5260f527922" path="/var/lib/kubelet/pods/6b06d932-79c5-4415-9b7e-f5260f527922/volumes"
Sep 11 13:24:09.875169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875089    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:24:09.875524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875493    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6b06d932-79c5-4415-9b7e-f5260f527922" containerName="maas-api"
Sep 11 13:24:09.875524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875507    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="6b06d932-79c5-4415-9b7e-f5260f527922" containerName="maas-api"
Sep 11 13:24:09.875524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875516    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:24:09.875524 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875522    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875545    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875551    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875556    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875561    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875570    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875575    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875627    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="6b06d932-79c5-4415-9b7e-f5260f527922" containerName="maas-api"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875635    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="5f63655f-1494-41c4-bc86-015548c12bfe" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875641    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3835415e-46f0-4c44-8f7b-8d58ff4eadae" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875647    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.875656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.875653    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="864069fa-eb4a-4cd0-b210-c89a99d45727" containerName="istio-proxy"
Sep 11 13:24:09.880373 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.880354    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:09.883741 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.883720    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-f78ffd60-openshift-default-dockercfg-xdcgt\""
Sep 11 13:24:09.918166 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:09.918134    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:24:10.016679 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016639    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016679 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016672    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016699    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016783    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016800    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016870    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016900    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wtb45\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.016943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016924    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.017265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.016951    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117695 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117659    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117695 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117695    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117759    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117776    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wtb45\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.117950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117909    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.117989    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118040    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118100    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118114    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118341 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118181    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118341 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118269    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118351    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.118924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.118901    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.120104 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.120088    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.120367 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.120350    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.132536 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.132472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.132633 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.132581    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wtb45\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45\") pod \"e2e-mt-f78ffd60-openshift-default-6886785888-7n76j\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") " pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.190354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.190319    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:10.508697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.508516    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:24:10.511491 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:24:10.511464    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod349c2479_43d8_4d03_830b_6d84fed5d59f.slice/crio-3a3341eb4ab47f048735f0d44f0660a3c804c3a327144573db436c3e952d4aea WatchSource:0}: Error finding container 3a3341eb4ab47f048735f0d44f0660a3c804c3a327144573db436c3e952d4aea: Status 404 returned error can't find the container with id 3a3341eb4ab47f048735f0d44f0660a3c804c3a327144573db436c3e952d4aea
Sep 11 13:24:10.513513 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.513479    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:10.513593 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.513541    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:10.513593 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:10.513569    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:11.490722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:11.490676    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerStarted","Data":"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"}
Sep 11 13:24:11.491170 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:11.490750    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerStarted","Data":"3a3341eb4ab47f048735f0d44f0660a3c804c3a327144573db436c3e952d4aea"}
Sep 11 13:24:11.539207 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:11.539154    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podStartSLOduration=2.539138969 podStartE2EDuration="2.539138969s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:11.538551785 +0000 UTC m=+1166.058589129" watchObservedRunningTime="2026-09-11 13:24:11.539138969 +0000 UTC m=+1166.059176315"
Sep 11 13:24:12.190507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:12.190474    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:12.195406 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:12.195379    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:12.496052 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:12.496023    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:12.496945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:12.496923    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:14.046668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:14.046634    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:25.353418 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:25.352681    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:28.473217 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.473183    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:24:28.486889 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.486849    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" podUID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" containerName="maas-api" containerID="cri-o://8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de" gracePeriod=30
Sep 11 13:24:28.551865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.551827    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:24:28.552164 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.552113    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" podUID="b813a496-6e34-4765-8e43-4f03ab789041" containerName="payload-pre-processing" containerID="cri-o://d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0" gracePeriod=30
Sep 11 13:24:28.815305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.815272    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:24:28.839192 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.839155    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:24:28.885666 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885641    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hg6mc\" (UniqueName: \"kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc\") pod \"b813a496-6e34-4765-8e43-4f03ab789041\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") "
Sep 11 13:24:28.885856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885726    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs\") pod \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") "
Sep 11 13:24:28.885856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885752    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca\") pod \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") "
Sep 11 13:24:28.885856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume\") pod \"b813a496-6e34-4765-8e43-4f03ab789041\" (UID: \"b813a496-6e34-4765-8e43-4f03ab789041\") "
Sep 11 13:24:28.885856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885851    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls\") pod \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") "
Sep 11 13:24:28.886081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.885942    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2fx9n\" (UniqueName: \"kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n\") pod \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\" (UID: \"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3\") "
Sep 11 13:24:28.886811 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.886769    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "b813a496-6e34-4765-8e43-4f03ab789041" (UID: "b813a496-6e34-4765-8e43-4f03ab789041"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:28.886977 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.886953    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" (UID: "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:24:28.888250 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.888221    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc" (OuterVolumeSpecName: "kube-api-access-hg6mc") pod "b813a496-6e34-4765-8e43-4f03ab789041" (UID: "b813a496-6e34-4765-8e43-4f03ab789041"). InnerVolumeSpecName "kube-api-access-hg6mc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:28.888401 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.888377    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" (UID: "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:28.889327 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.889298    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" (UID: "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:24:28.889457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.889435    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n" (OuterVolumeSpecName: "kube-api-access-2fx9n") pod "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" (UID: "8a1ab013-23ef-4f22-a39d-4a14bcb71bf3"). InnerVolumeSpecName "kube-api-access-2fx9n". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:24:28.986766 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986736    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.986924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986763    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2fx9n\" (UniqueName: \"kubernetes.io/projected/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-kube-api-access-2fx9n\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.986924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986783    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hg6mc\" (UniqueName: \"kubernetes.io/projected/b813a496-6e34-4765-8e43-4f03ab789041-kube-api-access-hg6mc\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.986924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986793    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.986924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986803    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:28.986924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:28.986813    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/b813a496-6e34-4765-8e43-4f03ab789041-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:24:29.590938 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.590898    2573 generic.go:356] "Generic (PLEG): container finished" podID="b813a496-6e34-4765-8e43-4f03ab789041" containerID="d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0" exitCode=0
Sep 11 13:24:29.591412 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.590949    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"
Sep 11 13:24:29.591412 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.590988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" event={"ID":"b813a496-6e34-4765-8e43-4f03ab789041","Type":"ContainerDied","Data":"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"}
Sep 11 13:24:29.591412 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.591061    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm" event={"ID":"b813a496-6e34-4765-8e43-4f03ab789041","Type":"ContainerDied","Data":"63faf22e77f41e29509911918724f32031ba621089d3c7a70c2adc3fdae3003d"}
Sep 11 13:24:29.591412 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.591096    2573 scope.go:117] "RemoveContainer" containerID="d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"
Sep 11 13:24:29.592660 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.592563    2573 generic.go:356] "Generic (PLEG): container finished" podID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" containerID="8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de" exitCode=0
Sep 11 13:24:29.592660 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.592614    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"
Sep 11 13:24:29.592660 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.592645    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" event={"ID":"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3","Type":"ContainerDied","Data":"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"}
Sep 11 13:24:29.592871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.592686    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x" event={"ID":"8a1ab013-23ef-4f22-a39d-4a14bcb71bf3","Type":"ContainerDied","Data":"f00675f19448f18a135508bfabf1c0fce7abadc5b69f700cd0a9ba695274fe94"}
Sep 11 13:24:29.601207 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.601066    2573 scope.go:117] "RemoveContainer" containerID="d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"
Sep 11 13:24:29.601426 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:24:29.601398    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0\": container with ID starting with d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0 not found: ID does not exist" containerID="d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"
Sep 11 13:24:29.601548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.601431    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0"} err="failed to get container status \"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0\": rpc error: code = NotFound desc = could not find container \"d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0\": container with ID starting with d47795c6ceffa0c65b25d242d639f84cc49e30b921d9966d886d689399f9e8d0 not found: ID does not exist"
Sep 11 13:24:29.601548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.601455    2573 scope.go:117] "RemoveContainer" containerID="8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"
Sep 11 13:24:29.610999 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.610982    2573 scope.go:117] "RemoveContainer" containerID="8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"
Sep 11 13:24:29.611267 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:24:29.611246    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de\": container with ID starting with 8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de not found: ID does not exist" containerID="8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"
Sep 11 13:24:29.611345 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.611271    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de"} err="failed to get container status \"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de\": rpc error: code = NotFound desc = could not find container \"8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de\": container with ID starting with 8410860b154e6ab96c5633a128f9ad7c698e5c5501cce05f1ae4561c14bda1de not found: ID does not exist"
Sep 11 13:24:29.618923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.618902    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:24:29.622846 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.622825    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-24931b-847d5f4b7d-xgp2x"]
Sep 11 13:24:29.633560 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.633539    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:24:29.637730 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.637695    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-inf-b-24931b-57564ff97d-6nmfm"]
Sep 11 13:24:29.972682 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.972609    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" path="/var/lib/kubelet/pods/8a1ab013-23ef-4f22-a39d-4a14bcb71bf3/volumes"
Sep 11 13:24:29.972989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:29.972976    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b813a496-6e34-4765-8e43-4f03ab789041" path="/var/lib/kubelet/pods/b813a496-6e34-4765-8e43-4f03ab789041/volumes"
Sep 11 13:24:31.606247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.606173    2573 generic.go:356] "Generic (PLEG): container finished" podID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerID="21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0" exitCode=0
Sep 11 13:24:31.606619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.606251    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerDied","Data":"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"}
Sep 11 13:24:31.606680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.606664    2573 scope.go:117] "RemoveContainer" containerID="21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"
Sep 11 13:24:31.607698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.607669    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:31.607831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.607767    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:31.607831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:31.607819    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:32.614197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:32.614161    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerStarted","Data":"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"}
Sep 11 13:24:32.871005 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:32.870926    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:33.190763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:33.190652    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:33.190903 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:33.190854    2573 patch_prober.go:28] interesting pod/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 13:24:33.190951 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:33.190901    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 13:24:34.191372 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:34.191337    2573 patch_prober.go:28] interesting pod/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 13:24:34.191764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:34.191394    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 13:24:35.191409 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:35.191378    2573 patch_prober.go:28] interesting pod/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 13:24:35.191845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:35.191438    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 13:24:36.191061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:36.191029    2573 patch_prober.go:28] interesting pod/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 13:24:36.191236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:36.191080    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 13:24:37.190919 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:37.190883    2573 patch_prober.go:28] interesting pod/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused" start-of-body=
Sep 11 13:24:37.191294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:37.190947    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.57:15021/healthz/ready\": dial tcp 10.134.0.57:15021: connect: connection refused"
Sep 11 13:24:38.194961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:38.194930    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:38.195501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:38.195227    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:38.195891 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:38.195873    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:24:39.548257 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548226    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:24:39.548613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548559    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b813a496-6e34-4765-8e43-4f03ab789041" containerName="payload-pre-processing"
Sep 11 13:24:39.548613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548571    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b813a496-6e34-4765-8e43-4f03ab789041" containerName="payload-pre-processing"
Sep 11 13:24:39.548613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548583    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" containerName="maas-api"
Sep 11 13:24:39.548613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548589    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" containerName="maas-api"
Sep 11 13:24:39.548774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548652    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b813a496-6e34-4765-8e43-4f03ab789041" containerName="payload-pre-processing"
Sep 11 13:24:39.548774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.548663    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8a1ab013-23ef-4f22-a39d-4a14bcb71bf3" containerName="maas-api"
Sep 11 13:24:39.550574 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.550558    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.560110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.560092    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w3-6ffb6a-openshift-default-dockercfg-2jwxs\""
Sep 11 13:24:39.604518 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.604486    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:24:39.693152 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693123    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693152 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693151    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693173    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693200    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693252    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-227d9\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693303    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693322    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693369    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.693501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.693391    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.794669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.794638    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.794669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.794669    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.794900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.794689    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.794900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.794752    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.794900 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.794792    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-227d9\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795071    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795086    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795149 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795131    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795185    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795218    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795338    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795590 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795448    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795590 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795559    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.795871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.795850    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.797010 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.796992    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.797407 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.797389    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.829654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.829599    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.831347 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.831329    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-227d9\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9\") pod \"e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") " pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:39.860872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:39.860846    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:40.159578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.159545    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:24:40.162324 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:24:40.162296    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7da9e048_26b0_4346_8c42_25024e0c5439.slice/crio-b4d72df8d5d48376ddadbb9c08d356fd69284499cb0b168e463c3dc0d6e51e0e WatchSource:0}: Error finding container b4d72df8d5d48376ddadbb9c08d356fd69284499cb0b168e463c3dc0d6e51e0e: Status 404 returned error can't find the container with id b4d72df8d5d48376ddadbb9c08d356fd69284499cb0b168e463c3dc0d6e51e0e
Sep 11 13:24:40.164466 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.164425    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:40.164548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.164501    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:40.164548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.164536    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:24:40.659039 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.659003    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerStarted","Data":"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"}
Sep 11 13:24:40.659390 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.659065    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerStarted","Data":"b4d72df8d5d48376ddadbb9c08d356fd69284499cb0b168e463c3dc0d6e51e0e"}
Sep 11 13:24:40.745888 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.745846    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podStartSLOduration=1.74583076 podStartE2EDuration="1.74583076s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:24:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:24:40.743440999 +0000 UTC m=+1195.263478344" watchObservedRunningTime="2026-09-11 13:24:40.74583076 +0000 UTC m=+1195.265868104"
Sep 11 13:24:40.861604 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:40.861583    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:41.867223 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:41.867183    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:42.669047 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:42.669024    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:42.670035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:42.670010    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:24:50.185146 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:50.185105    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:24:58.710846 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:58.710805    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 13:24:58.711254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:24:58.710858    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 13:25:00.763230 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.763196    2573 generic.go:356] "Generic (PLEG): container finished" podID="7da9e048-26b0-4346-8c42-25024e0c5439" containerID="0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac" exitCode=0
Sep 11 13:25:00.763644 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.763273    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerDied","Data":"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"}
Sep 11 13:25:00.763644 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.763639    2573 scope.go:117] "RemoveContainer" containerID="0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"
Sep 11 13:25:00.764602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.764576    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:00.764656 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.764641    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:00.764722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:00.764671    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:01.006318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:01.006289    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:25:01.776287 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:01.776252    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerStarted","Data":"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"}
Sep 11 13:25:01.861283 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:01.861237    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:25:01.861519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:01.861486    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:25:01.861679 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:01.861554    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 13:25:02.861967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:02.861926    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:25:02.862435 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:02.861978    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 13:25:03.861505 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:03.861461    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:25:03.861680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:03.861516    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 13:25:04.861519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:04.861485    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:25:04.861952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:04.861538    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 13:25:05.861841 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:05.861803    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused" start-of-body=
Sep 11 13:25:05.862352 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:05.861872    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.58:15021/healthz/ready\": dial tcp 10.134.0.58:15021: connect: connection refused"
Sep 11 13:25:06.865263 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:06.865234    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:25:06.865687 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:06.865495    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:25:06.866188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:06.866166    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:25:11.481556 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:11.481520    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:25:11.481959 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:11.481734    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy" containerID="cri-o://54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a" gracePeriod=30
Sep 11 13:25:16.731665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.731641    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:25:16.785233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785161    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785205    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785234    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785292    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785327    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wtb45\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785355    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785389    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785407    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:25:16.785444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785418    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785496    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket\") pod \"349c2479-43d8-4d03-830b-6d84fed5d59f\" (UID: \"349c2479-43d8-4d03-830b-6d84fed5d59f\") "
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785530    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785750    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data" (OuterVolumeSpecName: "istio-data") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785795    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785804    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.785871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785838    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.786190 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.785993    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:25:16.787851 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.787826    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:25:16.787981 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.787888    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token" (OuterVolumeSpecName: "istio-token") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:25:16.788052 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.788024    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:25:16.788113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.788068    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45" (OuterVolumeSpecName: "kube-api-access-wtb45") pod "349c2479-43d8-4d03-830b-6d84fed5d59f" (UID: "349c2479-43d8-4d03-830b-6d84fed5d59f"). InnerVolumeSpecName "kube-api-access-wtb45". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:25:16.856317 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.856282    2573 generic.go:356] "Generic (PLEG): container finished" podID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerID="54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a" exitCode=0
Sep 11 13:25:16.856502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.856331    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"
Sep 11 13:25:16.856502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.856368    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerDied","Data":"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"}
Sep 11 13:25:16.856502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.856434    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j" event={"ID":"349c2479-43d8-4d03-830b-6d84fed5d59f","Type":"ContainerDied","Data":"3a3341eb4ab47f048735f0d44f0660a3c804c3a327144573db436c3e952d4aea"}
Sep 11 13:25:16.856502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.856464    2573 scope.go:117] "RemoveContainer" containerID="54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"
Sep 11 13:25:16.865736 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.865684    2573 scope.go:117] "RemoveContainer" containerID="21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"
Sep 11 13:25:16.873941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.873927    2573 scope.go:117] "RemoveContainer" containerID="54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"
Sep 11 13:25:16.874210 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:25:16.874190    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a\": container with ID starting with 54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a not found: ID does not exist" containerID="54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"
Sep 11 13:25:16.874258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.874231    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a"} err="failed to get container status \"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a\": rpc error: code = NotFound desc = could not find container \"54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a\": container with ID starting with 54240c18c7a0f353a3294bfa95cf8620a8bb08409081f9328c9ba7bce441812a not found: ID does not exist"
Sep 11 13:25:16.874258 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.874247    2573 scope.go:117] "RemoveContainer" containerID="21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"
Sep 11 13:25:16.874474 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:25:16.874460    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0\": container with ID starting with 21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0 not found: ID does not exist" containerID="21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"
Sep 11 13:25:16.874533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.874475    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0"} err="failed to get container status \"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0\": rpc error: code = NotFound desc = could not find container \"21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0\": container with ID starting with 21e0abc3c2751b311a7d0f831e983287702717742078cc2eccb04ac2acd79fc0 not found: ID does not exist"
Sep 11 13:25:16.886986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.886969    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.886986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.886986    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wtb45\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-kube-api-access-wtb45\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.887122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.886997    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.887122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.887005    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.887122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.887013    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/349c2479-43d8-4d03-830b-6d84fed5d59f-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.887122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.887021    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.887122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.887028    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/349c2479-43d8-4d03-830b-6d84fed5d59f-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:25:16.901395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.901376    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:25:16.917828 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:16.917809    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-f78ffd60-openshift-default-6886785888-7n76j"]
Sep 11 13:25:17.978332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:17.978301    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" path="/var/lib/kubelet/pods/349c2479-43d8-4d03-830b-6d84fed5d59f/volumes"
Sep 11 13:25:20.087737 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:20.087686    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:25:31.565481 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565448    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565825    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565838    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565850    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565855    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565917    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.565930 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.565925    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="349c2479-43d8-4d03-830b-6d84fed5d59f" containerName="istio-proxy"
Sep 11 13:25:31.568669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.568651    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.572363 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:25:31.572336    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.572603 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:25:31.572579    2573 reflector.go:205] "Failed to watch" err="failed to list *v1.Secret: secrets \"e2e-shared-a-w2-e0ebd7-openshift-default-dockercfg-zj5w5\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot list resource \"secrets\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" logger="UnhandledError" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e0ebd7-openshift-default-dockercfg-zj5w5\"" type="*v1.Secret"
Sep 11 13:25:31.607321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607291    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607447 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607336    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607447 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607372    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607447 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607409    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607508    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607545    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607586    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmwqx\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607613    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.607889 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.607687    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.619831 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.619800    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:25:31.708668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708641    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.708790 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708672    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.708790 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708691    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmwqx\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.708790 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708756    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.708924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708819    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.708924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708871    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709031 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.708990    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709037    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709130    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709159    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709307 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709242    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709395    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.709811 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.709787    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.711090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.711070    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.711339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.711318    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.723295 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.723272    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmwqx\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:31.723748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:31.723731    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token\") pod \"e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") " pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:32.688501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:32.688464    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-shared-a-w2-e0ebd7-openshift-default-dockercfg-zj5w5\""
Sep 11 13:25:32.698101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:32.698075    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:32.999218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:32.999188    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:25:33.004501 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:25:33.004472    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod06f8f82d_8b86_42dd_bbe7_0eaac2d4c303.slice/crio-6443d705d48cf1e018fac4216a2f01b58e3348c5afc013f459dd758c91bd76a6 WatchSource:0}: Error finding container 6443d705d48cf1e018fac4216a2f01b58e3348c5afc013f459dd758c91bd76a6: Status 404 returned error can't find the container with id 6443d705d48cf1e018fac4216a2f01b58e3348c5afc013f459dd758c91bd76a6
Sep 11 13:25:33.006680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:33.006650    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:33.006770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:33.006729    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:33.006770 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:33.006759    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:33.946022 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:33.945988    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerStarted","Data":"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"}
Sep 11 13:25:33.946409 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:33.946048    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerStarted","Data":"6443d705d48cf1e018fac4216a2f01b58e3348c5afc013f459dd758c91bd76a6"}
Sep 11 13:25:34.037333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:34.037284    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podStartSLOduration=3.037269941 podStartE2EDuration="3.037269941s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:31 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:34.034796961 +0000 UTC m=+1248.554834304" watchObservedRunningTime="2026-09-11 13:25:34.037269941 +0000 UTC m=+1248.557307285"
Sep 11 13:25:34.698358 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:34.698266    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:34.703176 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:34.703152    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:34.951861 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:34.951782    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:34.952576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:34.952551    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:39.681008 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.680933    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:25:39.683432 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.683416    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.686619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.686600    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-shared-a-w2-e0ebd7\""
Sep 11 13:25:39.687645 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.687625    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-shared-a-w2-e0ebd7-dockercfg-sd646\""
Sep 11 13:25:39.768978 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.768940    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:25:39.784001 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.783972    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.784001 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.784004    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gx9ht\" (UniqueName: \"kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.808052 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.808027    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:25:39.810525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.810467    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.829122 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.829100    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:25:39.885195 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.885164    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvn8t\" (UniqueName: \"kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.885326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.885223    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.885326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.885254    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gx9ht\" (UniqueName: \"kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.885326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.885279    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.886208 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.886191    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.894197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.894175    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gx9ht\" (UniqueName: \"kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht\") pod \"payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") " pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:39.986321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.986245    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvn8t\" (UniqueName: \"kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.986321 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.986292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.986982 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.986964    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:39.994186 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:39.994169    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:40.003156 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.003138    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvn8t\" (UniqueName: \"kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t\") pod \"payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") " pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:40.121142 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.121112    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:40.273153 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.273129    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:25:40.275009 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:25:40.274985    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podab893501_c190_436a_a225_025b6d0b4d97.slice/crio-0785d11db3579af7d4e1ba44d0c5e36a1578feb0dbd01f0383ca3f5105020322 WatchSource:0}: Error finding container 0785d11db3579af7d4e1ba44d0c5e36a1578feb0dbd01f0383ca3f5105020322: Status 404 returned error can't find the container with id 0785d11db3579af7d4e1ba44d0c5e36a1578feb0dbd01f0383ca3f5105020322
Sep 11 13:25:40.414496 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.414474    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:25:40.416402 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:25:40.416378    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58902ff7_793e_4bc2_95e3_4e4a4d35662c.slice/crio-a7ccdc05439c0f3c26d2120561e324b77a46afadef844d8a475696b4907bdbd6 WatchSource:0}: Error finding container a7ccdc05439c0f3c26d2120561e324b77a46afadef844d8a475696b4907bdbd6: Status 404 returned error can't find the container with id a7ccdc05439c0f3c26d2120561e324b77a46afadef844d8a475696b4907bdbd6
Sep 11 13:25:40.986888 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.986847    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" event={"ID":"58902ff7-793e-4bc2-95e3-4e4a4d35662c","Type":"ContainerStarted","Data":"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"}
Sep 11 13:25:40.987345 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.986908    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" event={"ID":"58902ff7-793e-4bc2-95e3-4e4a4d35662c","Type":"ContainerStarted","Data":"a7ccdc05439c0f3c26d2120561e324b77a46afadef844d8a475696b4907bdbd6"}
Sep 11 13:25:40.987345 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.986938    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:40.988739 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.988692    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" event={"ID":"ab893501-c190-436a-a225-025b6d0b4d97","Type":"ContainerStarted","Data":"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"}
Sep 11 13:25:40.988858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.988755    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" event={"ID":"ab893501-c190-436a-a225-025b6d0b4d97","Type":"ContainerStarted","Data":"0785d11db3579af7d4e1ba44d0c5e36a1578feb0dbd01f0383ca3f5105020322"}
Sep 11 13:25:40.988858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:40.988837    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:41.022071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:41.022029    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" podStartSLOduration=2.022015489 podStartE2EDuration="2.022015489s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:41.019459529 +0000 UTC m=+1255.539496873" watchObservedRunningTime="2026-09-11 13:25:41.022015489 +0000 UTC m=+1255.542052867"
Sep 11 13:25:41.038087 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:41.038042    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" podStartSLOduration=2.038026194 podStartE2EDuration="2.038026194s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:25:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:25:41.037000808 +0000 UTC m=+1255.557038155" watchObservedRunningTime="2026-09-11 13:25:41.038026194 +0000 UTC m=+1255.558063540"
Sep 11 13:25:43.211095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:43.211063    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:25:51.996138 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:51.996096    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:25:51.996628 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:51.996275    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:25:54.057715 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.057635    2573 generic.go:356] "Generic (PLEG): container finished" podID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerID="56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432" exitCode=0
Sep 11 13:25:54.057715 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.057686    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerDied","Data":"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"}
Sep 11 13:25:54.058188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.058023    2573 scope.go:117] "RemoveContainer" containerID="56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"
Sep 11 13:25:54.060318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.059606    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:54.060318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.059785    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:54.060318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:54.059884    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:25:55.065611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:55.065574    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerStarted","Data":"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"}
Sep 11 13:25:55.698791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:55.698753    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:25:55.698996 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:55.698974    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 11 13:25:55.699043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:55.699022    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 11 13:25:56.699058 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:56.699022    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 11 13:25:56.699440 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:56.699085    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 11 13:25:57.699211 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:57.699173    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 11 13:25:57.699591 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:57.699233    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 11 13:25:58.698602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:58.698565    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 11 13:25:58.698813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:58.698626    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 11 13:25:59.699021 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:59.698981    2573 patch_prober.go:28] interesting pod/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused" start-of-body=
Sep 11 13:25:59.699392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:25:59.699051    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.59:15021/healthz/ready\": dial tcp 10.134.0.59:15021: connect: connection refused"
Sep 11 13:26:00.702597 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:00.702565    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:26:00.703049 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:00.702888    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:26:00.703518 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:00.703503    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:26:08.409487 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.409445    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:26:08.412910 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.412889    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.416546 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.416521    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-shared-b-w2-1eb525-serving-cert\""
Sep 11 13:26:08.437934 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.437909    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:26:08.455803 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.455779    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.455921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.455808    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nlc62\" (UniqueName: \"kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.455921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.455837    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.455921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.455855    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.557280 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.557241    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nlc62\" (UniqueName: \"kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.557477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.557292    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.557477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.557319    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.557477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.557461    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.557652 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:08.557524    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-serving-cert: secret "maas-api-e2e-shared-b-w2-1eb525-serving-cert" not found
Sep 11 13:26:08.557652 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:08.557608    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls podName:7671dc6e-de9f-4f02-883c-9ac69cd87a4f nodeName:}" failed. No retries permitted until 2026-09-11 13:26:09.057586296 +0000 UTC m=+1283.577623625 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls") pod "maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f") : secret "maas-api-e2e-shared-b-w2-1eb525-serving-cert" not found
Sep 11 13:26:08.558129 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.558107    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.559768 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.559750    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:08.581014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:08.580986    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nlc62\" (UniqueName: \"kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:09.062088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:09.062051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:09.062246 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:09.062209    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-serving-cert: secret "maas-api-e2e-shared-b-w2-1eb525-serving-cert" not found
Sep 11 13:26:09.062297 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:09.062286    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls podName:7671dc6e-de9f-4f02-883c-9ac69cd87a4f nodeName:}" failed. No retries permitted until 2026-09-11 13:26:10.062265847 +0000 UTC m=+1284.582303181 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls") pod "maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f") : secret "maas-api-e2e-shared-b-w2-1eb525-serving-cert" not found
Sep 11 13:26:10.072496 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:10.072434    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:10.074966 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:10.074937    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") " pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:10.224265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:10.224233    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:10.495649 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:10.495625    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:26:10.497610 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:10.497576    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7671dc6e_de9f_4f02_883c_9ac69cd87a4f.slice/crio-f45ee55376953dd02c7dcef51faa1a752f8e35c47e42d14edd52c990e2a3edc3 WatchSource:0}: Error finding container f45ee55376953dd02c7dcef51faa1a752f8e35c47e42d14edd52c990e2a3edc3: Status 404 returned error can't find the container with id f45ee55376953dd02c7dcef51faa1a752f8e35c47e42d14edd52c990e2a3edc3
Sep 11 13:26:10.499384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:10.499368    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:26:11.147589 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:11.147555    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" event={"ID":"7671dc6e-de9f-4f02-883c-9ac69cd87a4f","Type":"ContainerStarted","Data":"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"}
Sep 11 13:26:11.147945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:11.147600    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" event={"ID":"7671dc6e-de9f-4f02-883c-9ac69cd87a4f","Type":"ContainerStarted","Data":"f45ee55376953dd02c7dcef51faa1a752f8e35c47e42d14edd52c990e2a3edc3"}
Sep 11 13:26:11.147945 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:11.147695    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:11.177554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:11.177515    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" podStartSLOduration=2.593511438 podStartE2EDuration="3.177501221s" totalImagesPullingTime="583.989783ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:11.176018203 +0000 UTC m=+1285.696055548" watchObservedRunningTime="2026-09-11 13:26:11.177501221 +0000 UTC m=+1285.697538618"
Sep 11 13:26:12.703968 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:12.703938    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:26:17.158339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:17.158302    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:26:30.402264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:30.402224    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:26:35.333439 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.333398    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-b047d2"]
Sep 11 13:26:35.336074 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.336053    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:35.339035 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.339014    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6zc4w\""
Sep 11 13:26:35.345137 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.345111    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-b047d2"]
Sep 11 13:26:35.507065 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.507031    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vkjv\" (UniqueName: \"kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv\") pod \"test-curl-7693-b047d2\" (UID: \"4e6f959f-e1d6-4256-9a5c-4e3324d868f1\") " pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:35.607853 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.607778    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5vkjv\" (UniqueName: \"kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv\") pod \"test-curl-7693-b047d2\" (UID: \"4e6f959f-e1d6-4256-9a5c-4e3324d868f1\") " pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:35.620629 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.620598    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vkjv\" (UniqueName: \"kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv\") pod \"test-curl-7693-b047d2\" (UID: \"4e6f959f-e1d6-4256-9a5c-4e3324d868f1\") " pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:35.645808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.645783    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:35.924934 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:35.924910    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-b047d2"]
Sep 11 13:26:35.927399 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:35.927374    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4e6f959f_e1d6_4256_9a5c_4e3324d868f1.slice/crio-642117bf528b06c1bb873fce2a14e3c9766f6f002a519dad1e3ab1853b5c0efc WatchSource:0}: Error finding container 642117bf528b06c1bb873fce2a14e3c9766f6f002a519dad1e3ab1853b5c0efc: Status 404 returned error can't find the container with id 642117bf528b06c1bb873fce2a14e3c9766f6f002a519dad1e3ab1853b5c0efc
Sep 11 13:26:36.278632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:36.278595    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-b047d2" event={"ID":"4e6f959f-e1d6-4256-9a5c-4e3324d868f1","Type":"ContainerStarted","Data":"3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa"}
Sep 11 13:26:36.278632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:36.278638    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-b047d2" event={"ID":"4e6f959f-e1d6-4256-9a5c-4e3324d868f1","Type":"ContainerStarted","Data":"642117bf528b06c1bb873fce2a14e3c9766f6f002a519dad1e3ab1853b5c0efc"}
Sep 11 13:26:36.300102 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:36.300051    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7693-b047d2" podStartSLOduration=1.300031378 podStartE2EDuration="1.300031378s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:36.299651166 +0000 UTC m=+1310.819688511" watchObservedRunningTime="2026-09-11 13:26:36.300031378 +0000 UTC m=+1310.820068702"
Sep 11 13:26:36.955032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:36.954999    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-b047d2"]
Sep 11 13:26:36.964887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:36.964857    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-b047d2"]
Sep 11 13:26:37.085333 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.085293    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-b810a2"]
Sep 11 13:26:37.085782 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.085755    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" containerName="test-curl-7693-b047d2"
Sep 11 13:26:37.085782 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.085776    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" containerName="test-curl-7693-b047d2"
Sep 11 13:26:37.085962 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.085867    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" containerName="test-curl-7693-b047d2"
Sep 11 13:26:37.088229 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.088208    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:37.098444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.098421    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-b810a2"]
Sep 11 13:26:37.222200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.222168    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5cp2\" (UniqueName: \"kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2\") pod \"test-curl-7693-b810a2\" (UID: \"fc0b959a-d84a-4a43-a33d-059761866b27\") " pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:37.284135 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:37.284109    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7693-b047d2" container="test-curl-7693-b047d2"
Sep 11 13:26:37.323123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.323098    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5cp2\" (UniqueName: \"kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2\") pod \"test-curl-7693-b810a2\" (UID: \"fc0b959a-d84a-4a43-a33d-059761866b27\") " pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:37.333224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.333204    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5cp2\" (UniqueName: \"kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2\") pod \"test-curl-7693-b810a2\" (UID: \"fc0b959a-d84a-4a43-a33d-059761866b27\") " pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:37.397785 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.397676    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:37.667218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:37.667192    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-b810a2"]
Sep 11 13:26:37.668752 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:37.668697    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfc0b959a_d84a_4a43_a33d_059761866b27.slice/crio-a1624ab9c6f1c08d9b20aaa7bc231a5d3a39037bf6ef8f2b1440bd15fd5bde9c WatchSource:0}: Error finding container a1624ab9c6f1c08d9b20aaa7bc231a5d3a39037bf6ef8f2b1440bd15fd5bde9c: Status 404 returned error can't find the container with id a1624ab9c6f1c08d9b20aaa7bc231a5d3a39037bf6ef8f2b1440bd15fd5bde9c
Sep 11 13:26:38.291329 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.291287    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-b810a2" event={"ID":"fc0b959a-d84a-4a43-a33d-059761866b27","Type":"ContainerStarted","Data":"468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51"}
Sep 11 13:26:38.291329 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.291333    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-b810a2" event={"ID":"fc0b959a-d84a-4a43-a33d-059761866b27","Type":"ContainerStarted","Data":"a1624ab9c6f1c08d9b20aaa7bc231a5d3a39037bf6ef8f2b1440bd15fd5bde9c"}
Sep 11 13:26:38.291823 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.291428    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7693-b047d2" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" containerName="test-curl-7693-b047d2" containerID="cri-o://3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa" gracePeriod=2
Sep 11 13:26:38.307847 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.307800    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7693-b810a2" podStartSLOduration=1.307786345 podStartE2EDuration="1.307786345s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:38.305907517 +0000 UTC m=+1312.825944863" watchObservedRunningTime="2026-09-11 13:26:38.307786345 +0000 UTC m=+1312.827823690"
Sep 11 13:26:38.875325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.875293    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-b810a2"]
Sep 11 13:26:38.879485 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.879461    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-b810a2"]
Sep 11 13:26:38.987121 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.987094    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-e8fe16"]
Sep 11 13:26:38.987548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.987530    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" containerName="test-curl-7693-b810a2"
Sep 11 13:26:38.987636 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.987551    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" containerName="test-curl-7693-b810a2"
Sep 11 13:26:38.987691 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.987635    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" containerName="test-curl-7693-b810a2"
Sep 11 13:26:38.989563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:38.989544    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:39.001392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.001369    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-e8fe16"]
Sep 11 13:26:39.141750 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.141720    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s7rxv\" (UniqueName: \"kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv\") pod \"test-curl-7693-e8fe16\" (UID: \"235d13eb-9cfb-4661-9008-1b0ca9d62f53\") " pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:39.242332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.242244    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s7rxv\" (UniqueName: \"kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv\") pod \"test-curl-7693-e8fe16\" (UID: \"235d13eb-9cfb-4661-9008-1b0ca9d62f53\") " pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:39.250748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.250720    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s7rxv\" (UniqueName: \"kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv\") pod \"test-curl-7693-e8fe16\" (UID: \"235d13eb-9cfb-4661-9008-1b0ca9d62f53\") " pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:39.296192 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:39.296164    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7693-b810a2" container="test-curl-7693-b810a2"
Sep 11 13:26:39.298025 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.298010    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:39.576972 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:39.576943    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-e8fe16"]
Sep 11 13:26:39.578376 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:39.578346    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod235d13eb_9cfb_4661_9008_1b0ca9d62f53.slice/crio-f57ba6cb96baa213c26ba24c8d53da5938709fa13213c27eb04e0ad68e87452a WatchSource:0}: Error finding container f57ba6cb96baa213c26ba24c8d53da5938709fa13213c27eb04e0ad68e87452a: Status 404 returned error can't find the container with id f57ba6cb96baa213c26ba24c8d53da5938709fa13213c27eb04e0ad68e87452a
Sep 11 13:26:40.304140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.304107    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-e8fe16" event={"ID":"235d13eb-9cfb-4661-9008-1b0ca9d62f53","Type":"ContainerStarted","Data":"031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f"}
Sep 11 13:26:40.304476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.304149    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-e8fe16" event={"ID":"235d13eb-9cfb-4661-9008-1b0ca9d62f53","Type":"ContainerStarted","Data":"f57ba6cb96baa213c26ba24c8d53da5938709fa13213c27eb04e0ad68e87452a"}
Sep 11 13:26:40.304476 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.304343    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7693-b810a2" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" containerName="test-curl-7693-b810a2" containerID="cri-o://468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51" gracePeriod=2
Sep 11 13:26:40.321774 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.321723    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7693-e8fe16" podStartSLOduration=2.321688542 podStartE2EDuration="2.321688542s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:40.319776504 +0000 UTC m=+1314.839813847" watchObservedRunningTime="2026-09-11 13:26:40.321688542 +0000 UTC m=+1314.841725887"
Sep 11 13:26:40.431104 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.431076    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:40.554880 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.554787    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vkjv\" (UniqueName: \"kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv\") pod \"4e6f959f-e1d6-4256-9a5c-4e3324d868f1\" (UID: \"4e6f959f-e1d6-4256-9a5c-4e3324d868f1\") "
Sep 11 13:26:40.556941 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.556907    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv" (OuterVolumeSpecName: "kube-api-access-5vkjv") pod "4e6f959f-e1d6-4256-9a5c-4e3324d868f1" (UID: "4e6f959f-e1d6-4256-9a5c-4e3324d868f1"). InnerVolumeSpecName "kube-api-access-5vkjv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:40.655872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.655835    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vkjv\" (UniqueName: \"kubernetes.io/projected/4e6f959f-e1d6-4256-9a5c-4e3324d868f1-kube-api-access-5vkjv\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:26:40.896453 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.896364    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-e8fe16"]
Sep 11 13:26:40.903579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:40.903551    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-e8fe16"]
Sep 11 13:26:41.026402 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.026364    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7693-77056e"]
Sep 11 13:26:41.026779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.026765    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" containerName="test-curl-7693-e8fe16"
Sep 11 13:26:41.026833 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.026782    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" containerName="test-curl-7693-e8fe16"
Sep 11 13:26:41.026870 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.026846    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" containerName="test-curl-7693-e8fe16"
Sep 11 13:26:41.029023 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.029007    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:41.039695 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.039671    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-77056e"]
Sep 11 13:26:41.160872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.160763    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jmkvf\" (UniqueName: \"kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf\") pod \"test-curl-7693-77056e\" (UID: \"0bc7f823-6bde-4a46-802b-359fd0258be5\") " pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:41.262094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.262051    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jmkvf\" (UniqueName: \"kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf\") pod \"test-curl-7693-77056e\" (UID: \"0bc7f823-6bde-4a46-802b-359fd0258be5\") " pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:41.273606 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.273569    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jmkvf\" (UniqueName: \"kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf\") pod \"test-curl-7693-77056e\" (UID: \"0bc7f823-6bde-4a46-802b-359fd0258be5\") " pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:41.310090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.310048    2573 generic.go:356] "Generic (PLEG): container finished" podID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" containerID="3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa" exitCode=137
Sep 11 13:26:41.310090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.310093    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:41.310593 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.310148    2573 scope.go:117] "RemoveContainer" containerID="3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa"
Sep 11 13:26:41.310593 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:41.310467    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7693-e8fe16" container="test-curl-7693-e8fe16"
Sep 11 13:26:41.319938 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.319917    2573 scope.go:117] "RemoveContainer" containerID="3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa"
Sep 11 13:26:41.320226 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:41.320203    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa\": container with ID starting with 3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa not found: ID does not exist" containerID="3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa"
Sep 11 13:26:41.320294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.320240    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa"} err="failed to get container status \"3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa\": rpc error: code = NotFound desc = could not find container \"3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa\": container with ID starting with 3cea00792a65c882c2a6edfae08cae05ba5ef945bceb8f0bbddc353d4ed8bcaa not found: ID does not exist"
Sep 11 13:26:41.322284 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:41.322261    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-b047d2\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" pod="opendatahub/test-curl-7693-b047d2"
Sep 11 13:26:41.338461 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.338431    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:41.611193 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.611166    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7693-77056e"]
Sep 11 13:26:41.612237 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:41.612210    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0bc7f823_6bde_4a46_802b_359fd0258be5.slice/crio-e725eb01ce1f6718508bc1ade4fd3a28206218134bf15a8629dd5f4953b003b1 WatchSource:0}: Error finding container e725eb01ce1f6718508bc1ade4fd3a28206218134bf15a8629dd5f4953b003b1: Status 404 returned error can't find the container with id e725eb01ce1f6718508bc1ade4fd3a28206218134bf15a8629dd5f4953b003b1
Sep 11 13:26:41.972379 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:41.972301    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e6f959f-e1d6-4256-9a5c-4e3324d868f1" path="/var/lib/kubelet/pods/4e6f959f-e1d6-4256-9a5c-4e3324d868f1/volumes"
Sep 11 13:26:42.320697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.320671    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-77056e" event={"ID":"0bc7f823-6bde-4a46-802b-359fd0258be5","Type":"ContainerStarted","Data":"ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c"}
Sep 11 13:26:42.321038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.320733    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7693-77056e" event={"ID":"0bc7f823-6bde-4a46-802b-359fd0258be5","Type":"ContainerStarted","Data":"e725eb01ce1f6718508bc1ade4fd3a28206218134bf15a8629dd5f4953b003b1"}
Sep 11 13:26:42.321696 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.321672    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7693-e8fe16" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" containerName="test-curl-7693-e8fe16" containerID="cri-o://031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f" gracePeriod=2
Sep 11 13:26:42.335830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.335786    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7693-77056e" podStartSLOduration=1.335771021 podStartE2EDuration="1.335771021s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:42.334524124 +0000 UTC m=+1316.854561459" watchObservedRunningTime="2026-09-11 13:26:42.335771021 +0000 UTC m=+1316.855808365"
Sep 11 13:26:42.475852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.475829    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:42.571965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.571873    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5cp2\" (UniqueName: \"kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2\") pod \"fc0b959a-d84a-4a43-a33d-059761866b27\" (UID: \"fc0b959a-d84a-4a43-a33d-059761866b27\") "
Sep 11 13:26:42.573845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.573813    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2" (OuterVolumeSpecName: "kube-api-access-d5cp2") pod "fc0b959a-d84a-4a43-a33d-059761866b27" (UID: "fc0b959a-d84a-4a43-a33d-059761866b27"). InnerVolumeSpecName "kube-api-access-d5cp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:42.672618 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.672583    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5cp2\" (UniqueName: \"kubernetes.io/projected/fc0b959a-d84a-4a43-a33d-059761866b27-kube-api-access-d5cp2\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:26:42.889607 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.889521    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7693-77056e"]
Sep 11 13:26:42.896196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:42.896164    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7693-77056e"]
Sep 11 13:26:43.328066 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.328027    2573 generic.go:356] "Generic (PLEG): container finished" podID="fc0b959a-d84a-4a43-a33d-059761866b27" containerID="468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51" exitCode=137
Sep 11 13:26:43.328559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.328078    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:43.328559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.328115    2573 scope.go:117] "RemoveContainer" containerID="468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51"
Sep 11 13:26:43.328559 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:43.328351    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7693-77056e" container="test-curl-7693-77056e"
Sep 11 13:26:43.337265 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.337249    2573 scope.go:117] "RemoveContainer" containerID="468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51"
Sep 11 13:26:43.337515 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:43.337500    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51\": container with ID starting with 468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51 not found: ID does not exist" containerID="468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51"
Sep 11 13:26:43.337583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.337522    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51"} err="failed to get container status \"468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51\": rpc error: code = NotFound desc = could not find container \"468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51\": container with ID starting with 468f1948f90eacda20958e0239cd2f089fdf566ae4bf3b7b174be3615074ae51 not found: ID does not exist"
Sep 11 13:26:43.338812 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:43.338790    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-b810a2\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:43.754314 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.754283    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:26:43.754746 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.754730    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" containerName="test-curl-7693-77056e"
Sep 11 13:26:43.754746 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.754746    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" containerName="test-curl-7693-77056e"
Sep 11 13:26:43.754883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.754816    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" containerName="test-curl-7693-77056e"
Sep 11 13:26:43.759500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.759474    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.761790 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.761769    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-mt-3d5dcb7b-openshift-default-dockercfg-g6skc\""
Sep 11 13:26:43.773028 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.773006    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:26:43.780104 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780197 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780131    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780200    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780239    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780267    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5jh4s\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780287    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780309    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780327    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.780473 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.780351    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.800123 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:43.800087    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-b810a2\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" pod="opendatahub/test-curl-7693-b810a2"
Sep 11 13:26:43.881559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881530    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881559 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881562    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881584    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881606    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5jh4s\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881630    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881664    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881690    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.881834 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881735    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.881836    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.882086    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882312 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.882285    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.882371    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.882463    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.882477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.882472    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.884482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.884462    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.884798 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.884777    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.891824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.891799    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.891934 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.891840    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5jh4s\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s\") pod \"e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") " pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:43.972582 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:43.972545    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fc0b959a-d84a-4a43-a33d-059761866b27" path="/var/lib/kubelet/pods/fc0b959a-d84a-4a43-a33d-059761866b27/volumes"
Sep 11 13:26:44.069857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.069784    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:44.335870 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.335840    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7693-77056e" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" containerName="test-curl-7693-77056e" containerID="cri-o://ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c" gracePeriod=2
Sep 11 13:26:44.368338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.368316    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:26:44.383603 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:44.383575    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d81b8dc_6de1_4977_b069_fc711009d86d.slice/crio-f0ba57c7e8bd57a7fc428aa14e9a8fcefe368352c7e31445414acda46891bc00 WatchSource:0}: Error finding container f0ba57c7e8bd57a7fc428aa14e9a8fcefe368352c7e31445414acda46891bc00: Status 404 returned error can't find the container with id f0ba57c7e8bd57a7fc428aa14e9a8fcefe368352c7e31445414acda46891bc00
Sep 11 13:26:44.386008 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.385974    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:26:44.386077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.386045    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:26:44.386077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.386073    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:26:44.484312 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.484285    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:44.587599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.587516    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s7rxv\" (UniqueName: \"kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv\") pod \"235d13eb-9cfb-4661-9008-1b0ca9d62f53\" (UID: \"235d13eb-9cfb-4661-9008-1b0ca9d62f53\") "
Sep 11 13:26:44.589535 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.589509    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv" (OuterVolumeSpecName: "kube-api-access-s7rxv") pod "235d13eb-9cfb-4661-9008-1b0ca9d62f53" (UID: "235d13eb-9cfb-4661-9008-1b0ca9d62f53"). InnerVolumeSpecName "kube-api-access-s7rxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:44.688873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:44.688842    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s7rxv\" (UniqueName: \"kubernetes.io/projected/235d13eb-9cfb-4661-9008-1b0ca9d62f53-kube-api-access-s7rxv\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:26:45.342942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.342907    2573 generic.go:356] "Generic (PLEG): container finished" podID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" containerID="031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f" exitCode=137
Sep 11 13:26:45.342942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.342945    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:45.343386 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.342997    2573 scope.go:117] "RemoveContainer" containerID="031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f"
Sep 11 13:26:45.344801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.344767    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerStarted","Data":"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"}
Sep 11 13:26:45.344902 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.344809    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerStarted","Data":"f0ba57c7e8bd57a7fc428aa14e9a8fcefe368352c7e31445414acda46891bc00"}
Sep 11 13:26:45.352722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.352673    2573 scope.go:117] "RemoveContainer" containerID="031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f"
Sep 11 13:26:45.352964 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:45.352948    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f\": container with ID starting with 031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f not found: ID does not exist" containerID="031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f"
Sep 11 13:26:45.353013 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.352974    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f"} err="failed to get container status \"031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f\": rpc error: code = NotFound desc = could not find container \"031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f\": container with ID starting with 031a32bd85955538c6ca4f9e1beada651dd5d0ee6a1b96d2a18a5757237b354f not found: ID does not exist"
Sep 11 13:26:45.372131 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:45.372093    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-e8fe16\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:45.372907 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.372869    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podStartSLOduration=2.372854803 podStartE2EDuration="2.372854803s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:45.369411912 +0000 UTC m=+1319.889449258" watchObservedRunningTime="2026-09-11 13:26:45.372854803 +0000 UTC m=+1319.892892147"
Sep 11 13:26:45.971185 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:45.971150    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-e8fe16\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" pod="opendatahub/test-curl-7693-e8fe16"
Sep 11 13:26:45.973259 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:45.973232    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="235d13eb-9cfb-4661-9008-1b0ca9d62f53" path="/var/lib/kubelet/pods/235d13eb-9cfb-4661-9008-1b0ca9d62f53/volumes"
Sep 11 13:26:46.070640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.070620    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:46.075986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.075965    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:46.351408 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.351382    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:46.352585 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.352547    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:26:46.528760 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.528736    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:46.606473 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.606396    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jmkvf\" (UniqueName: \"kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf\") pod \"0bc7f823-6bde-4a46-802b-359fd0258be5\" (UID: \"0bc7f823-6bde-4a46-802b-359fd0258be5\") "
Sep 11 13:26:46.608439 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.608414    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf" (OuterVolumeSpecName: "kube-api-access-jmkvf") pod "0bc7f823-6bde-4a46-802b-359fd0258be5" (UID: "0bc7f823-6bde-4a46-802b-359fd0258be5"). InnerVolumeSpecName "kube-api-access-jmkvf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:26:46.707613 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:46.707584    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jmkvf\" (UniqueName: \"kubernetes.io/projected/0bc7f823-6bde-4a46-802b-359fd0258be5-kube-api-access-jmkvf\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:26:47.357963 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.357925    2573 generic.go:356] "Generic (PLEG): container finished" podID="0bc7f823-6bde-4a46-802b-359fd0258be5" containerID="ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c" exitCode=137
Sep 11 13:26:47.358359 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.357972    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:47.358359 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.358033    2573 scope.go:117] "RemoveContainer" containerID="ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c"
Sep 11 13:26:47.367077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.367059    2573 scope.go:117] "RemoveContainer" containerID="ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c"
Sep 11 13:26:47.367352 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:47.367334    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c\": container with ID starting with ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c not found: ID does not exist" containerID="ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c"
Sep 11 13:26:47.367418 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.367361    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c"} err="failed to get container status \"ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c\": rpc error: code = NotFound desc = could not find container \"ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c\": container with ID starting with ec3493a1d072f85d2101f79bc3e05cd8628e4ff19ef83d0b400c969d599aec8c not found: ID does not exist"
Sep 11 13:26:47.371037 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:26:47.371012    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7693-77056e\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" pod="opendatahub/test-curl-7693-77056e"
Sep 11 13:26:47.972526 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:47.972486    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0bc7f823-6bde-4a46-802b-359fd0258be5" path="/var/lib/kubelet/pods/0bc7f823-6bde-4a46-802b-359fd0258be5/volumes"
Sep 11 13:26:53.434921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.434887    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:26:53.439627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.439605    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.443293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.443269    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-mt-3d5dcb7b-dockercfg-6qdl9\""
Sep 11 13:26:53.443421 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.443318    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-mt-3d5dcb7b\""
Sep 11 13:26:53.455120 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.455095    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:26:53.467469 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.467431    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xcfqp\" (UniqueName: \"kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.467648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.467604    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.568297 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.568264    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.568447 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.568317    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xcfqp\" (UniqueName: \"kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.569006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.568986    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.577920 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.577897    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xcfqp\" (UniqueName: \"kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp\") pod \"payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") " pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:53.750242 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:53.750210    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:54.023218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:54.023190    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:26:54.024538 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:54.024514    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8037cd8b_c956_47cf_82f8_7c28985a4a31.slice/crio-b66643350dce2d4d5defc407288c722c7700a6e79f8e2f4ed1fec6fc63a3288c WatchSource:0}: Error finding container b66643350dce2d4d5defc407288c722c7700a6e79f8e2f4ed1fec6fc63a3288c: Status 404 returned error can't find the container with id b66643350dce2d4d5defc407288c722c7700a6e79f8e2f4ed1fec6fc63a3288c
Sep 11 13:26:54.398413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:54.398307    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" event={"ID":"8037cd8b-c956-47cf-82f8-7c28985a4a31","Type":"ContainerStarted","Data":"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"}
Sep 11 13:26:54.398413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:54.398383    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:26:54.398413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:54.398406    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" event={"ID":"8037cd8b-c956-47cf-82f8-7c28985a4a31","Type":"ContainerStarted","Data":"b66643350dce2d4d5defc407288c722c7700a6e79f8e2f4ed1fec6fc63a3288c"}
Sep 11 13:26:54.420815 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:54.420766    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" podStartSLOduration=1.42075075 podStartE2EDuration="1.42075075s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:54.418345504 +0000 UTC m=+1328.938382849" watchObservedRunningTime="2026-09-11 13:26:54.42075075 +0000 UTC m=+1328.940788096"
Sep 11 13:26:58.189175 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.189145    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:26:58.195316 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.193630    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.208523 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.208505    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"kube-root-ca.crt\""
Sep 11 13:26:58.208654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.208536    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"openshift-service-ca.crt\""
Sep 11 13:26:58.208654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.208506    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"auth-test-model-12f386-kserve-self-signed-certs\""
Sep 11 13:26:58.215481 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.215461    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:26:58.216089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.216070    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"default-dockercfg-285dr\""
Sep 11 13:26:58.309171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309147    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.309271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309179    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.309271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309216    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.309393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309356    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.309454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309393    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jlxlg\" (UniqueName: \"kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.309454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.309441    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.410588 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410552    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.410810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410640    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.410810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410724    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.410810 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410786    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jlxlg\" (UniqueName: \"kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.411004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410824    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.411004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.410884    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.411281 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.411252    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.411416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.411388    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.411539 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.411438    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.413415 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.413386    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.413859 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.413832    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.441307 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.441244    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jlxlg\" (UniqueName: \"kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg\") pod \"auth-test-model-12f386-kserve-6b67d9d6b7-stxbc\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") " pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.441678 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.441657    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:26:58.504874 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.504832    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:26:58.798946 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:58.798924    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:26:58.801340 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:26:58.801303    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddf17393e_15ad_44a7_bba2_90f5b242da7f.slice/crio-92cec6647f40377317ffb992d160c160be959fb16edb19b5c008c4ed15172504 WatchSource:0}: Error finding container 92cec6647f40377317ffb992d160c160be959fb16edb19b5c008c4ed15172504: Status 404 returned error can't find the container with id 92cec6647f40377317ffb992d160c160be959fb16edb19b5c008c4ed15172504
Sep 11 13:26:59.426348 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:59.426315    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" event={"ID":"df17393e-15ad-44a7-bba2-90f5b242da7f","Type":"ContainerStarted","Data":"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"}
Sep 11 13:26:59.426698 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:59.426356    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" event={"ID":"df17393e-15ad-44a7-bba2-90f5b242da7f","Type":"ContainerStarted","Data":"92cec6647f40377317ffb992d160c160be959fb16edb19b5c008c4ed15172504"}
Sep 11 13:26:59.446094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:26:59.446040    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" podStartSLOduration=1.446026772 podStartE2EDuration="1.446026772s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:26:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:26:59.443963116 +0000 UTC m=+1333.964000463" watchObservedRunningTime="2026-09-11 13:26:59.446026772 +0000 UTC m=+1333.966064116"
Sep 11 13:27:05.406988 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:05.406906    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:27:08.487442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.487415    2573 generic.go:356] "Generic (PLEG): container finished" podID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerID="09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656" exitCode=0
Sep 11 13:27:08.487868 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.487494    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerDied","Data":"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"}
Sep 11 13:27:08.487912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.487901    2573 scope.go:117] "RemoveContainer" containerID="09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"
Sep 11 13:27:08.488973 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.488938    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:27:08.489057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.489002    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:27:08.489057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.489033    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:27:08.505462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.505444    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:27:08.505545 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.505480    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:27:08.520012 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:08.519989    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:27:09.495174 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:09.495139    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerStarted","Data":"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"}
Sep 11 13:27:09.506583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:09.506555    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:27:10.070313 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:10.070278    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:27:10.070494 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:10.070470    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:27:10.070542 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:10.070515    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 13:27:11.070898 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:11.070862    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:27:11.071309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:11.070936    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 13:27:12.070434 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:12.070395    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:27:12.070603 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:12.070448    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 13:27:12.217538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:12.217510    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:27:13.070688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:13.070650    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:27:13.070886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:13.070732    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 13:27:14.070288 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:14.070248    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused" start-of-body=
Sep 11 13:27:14.070780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:14.070300    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.67:15021/healthz/ready\": dial tcp 10.134.0.67:15021: connect: connection refused"
Sep 11 13:27:15.074830 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:15.074792    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:27:15.075289 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:15.075053    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:27:15.075687 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:15.075670    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:27:26.525129 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:26.525090    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:27:43.699700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:27:43.699667    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:28:00.897748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:00.897698    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:28:17.449980 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.449941    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:28:17.451044 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.450259    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" podUID="8037cd8b-c956-47cf-82f8-7c28985a4a31" containerName="payload-pre-processing" containerID="cri-o://6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed" gracePeriod=30
Sep 11 13:28:17.695867 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.695842    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:28:17.836365 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.836333    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume\") pod \"8037cd8b-c956-47cf-82f8-7c28985a4a31\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") "
Sep 11 13:28:17.836551 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.836521    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xcfqp\" (UniqueName: \"kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp\") pod \"8037cd8b-c956-47cf-82f8-7c28985a4a31\" (UID: \"8037cd8b-c956-47cf-82f8-7c28985a4a31\") "
Sep 11 13:28:17.836720 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.836679    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8037cd8b-c956-47cf-82f8-7c28985a4a31" (UID: "8037cd8b-c956-47cf-82f8-7c28985a4a31"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:28:17.836927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.836910    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8037cd8b-c956-47cf-82f8-7c28985a4a31-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.838638 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.838609    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp" (OuterVolumeSpecName: "kube-api-access-xcfqp") pod "8037cd8b-c956-47cf-82f8-7c28985a4a31" (UID: "8037cd8b-c956-47cf-82f8-7c28985a4a31"). InnerVolumeSpecName "kube-api-access-xcfqp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:28:17.856527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.856498    2573 generic.go:356] "Generic (PLEG): container finished" podID="8037cd8b-c956-47cf-82f8-7c28985a4a31" containerID="6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed" exitCode=0
Sep 11 13:28:17.856669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.856561    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"
Sep 11 13:28:17.856669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.856580    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" event={"ID":"8037cd8b-c956-47cf-82f8-7c28985a4a31","Type":"ContainerDied","Data":"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"}
Sep 11 13:28:17.856669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.856647    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh" event={"ID":"8037cd8b-c956-47cf-82f8-7c28985a4a31","Type":"ContainerDied","Data":"b66643350dce2d4d5defc407288c722c7700a6e79f8e2f4ed1fec6fc63a3288c"}
Sep 11 13:28:17.856862 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.856678    2573 scope.go:117] "RemoveContainer" containerID="6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"
Sep 11 13:28:17.866599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.866585    2573 scope.go:117] "RemoveContainer" containerID="6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"
Sep 11 13:28:17.866886 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:28:17.866866    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed\": container with ID starting with 6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed not found: ID does not exist" containerID="6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"
Sep 11 13:28:17.866956 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.866894    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed"} err="failed to get container status \"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed\": rpc error: code = NotFound desc = could not find container \"6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed\": container with ID starting with 6de0b3bfd2461e727b2fc5136be0ba03315dce2708865b052761b40ace7bfaed not found: ID does not exist"
Sep 11 13:28:17.882073 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.882052    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:28:17.885865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.885844    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-mt-3d5dcb7b-7d5db9cfc8-jlzlh"]
Sep 11 13:28:17.937929 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.937897    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xcfqp\" (UniqueName: \"kubernetes.io/projected/8037cd8b-c956-47cf-82f8-7c28985a4a31-kube-api-access-xcfqp\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:28:17.972735 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:17.972694    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8037cd8b-c956-47cf-82f8-7c28985a4a31" path="/var/lib/kubelet/pods/8037cd8b-c956-47cf-82f8-7c28985a4a31/volumes"
Sep 11 13:28:21.698517 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:21.698482    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:28:38.099731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:38.099685    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:28:54.626296 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:28:54.626243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:29:06.276908 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.276878    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-d785ad"]
Sep 11 13:29:06.277264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.277243    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8037cd8b-c956-47cf-82f8-7c28985a4a31" containerName="payload-pre-processing"
Sep 11 13:29:06.277264 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.277254    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8037cd8b-c956-47cf-82f8-7c28985a4a31" containerName="payload-pre-processing"
Sep 11 13:29:06.277337 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.277322    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8037cd8b-c956-47cf-82f8-7c28985a4a31" containerName="payload-pre-processing"
Sep 11 13:29:06.281616 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.281600    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:06.284605 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.284581    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6zc4w\""
Sep 11 13:29:06.297500 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.297475    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-d785ad"]
Sep 11 13:29:06.364174 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.364142    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kb5tj\" (UniqueName: \"kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj\") pod \"test-curl-7696-d785ad\" (UID: \"ff008283-df6e-4300-b6d1-a49776e87e37\") " pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:06.465099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.465069    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kb5tj\" (UniqueName: \"kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj\") pod \"test-curl-7696-d785ad\" (UID: \"ff008283-df6e-4300-b6d1-a49776e87e37\") " pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:06.476304 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.476273    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kb5tj\" (UniqueName: \"kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj\") pod \"test-curl-7696-d785ad\" (UID: \"ff008283-df6e-4300-b6d1-a49776e87e37\") " pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:06.590860 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.590795    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:06.875124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:06.875101    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-d785ad"]
Sep 11 13:29:06.877109 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:29:06.877084    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff008283_df6e_4300_b6d1_a49776e87e37.slice/crio-9956e6be4f9cb04eece671796376beafb2784e64d9ddb2c35faa5342b247b2f4 WatchSource:0}: Error finding container 9956e6be4f9cb04eece671796376beafb2784e64d9ddb2c35faa5342b247b2f4: Status 404 returned error can't find the container with id 9956e6be4f9cb04eece671796376beafb2784e64d9ddb2c35faa5342b247b2f4
Sep 11 13:29:07.124772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:07.124675    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-d785ad" event={"ID":"ff008283-df6e-4300-b6d1-a49776e87e37","Type":"ContainerStarted","Data":"10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199"}
Sep 11 13:29:07.124772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:07.124747    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-d785ad" event={"ID":"ff008283-df6e-4300-b6d1-a49776e87e37","Type":"ContainerStarted","Data":"9956e6be4f9cb04eece671796376beafb2784e64d9ddb2c35faa5342b247b2f4"}
Sep 11 13:29:07.148493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:07.148440    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-d785ad" podStartSLOduration=1.14842641 podStartE2EDuration="1.14842641s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:29:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:29:07.145592427 +0000 UTC m=+1461.665629771" watchObservedRunningTime="2026-09-11 13:29:07.14842641 +0000 UTC m=+1461.668463755"
Sep 11 13:29:07.976275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:07.976196    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-d785ad"]
Sep 11 13:29:07.976275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:07.976239    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-d785ad"]
Sep 11 13:29:08.130631 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:08.130600    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-d785ad" container="test-curl-7696-d785ad"
Sep 11 13:29:09.136159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:09.136112    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-d785ad" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" containerName="test-curl-7696-d785ad" containerID="cri-o://10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199" gracePeriod=2
Sep 11 13:29:09.141388 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:09.141358    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-d785ad\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:09.228773 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:09.228746    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:29:10.459463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:10.459429    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:29:10.561205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:10.561165    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:29:10.561405 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:10.561378    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" containerID="cri-o://f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888" gracePeriod=30
Sep 11 13:29:11.299200 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:11.299178    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:11.412845 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:11.412783    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kb5tj\" (UniqueName: \"kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj\") pod \"ff008283-df6e-4300-b6d1-a49776e87e37\" (UID: \"ff008283-df6e-4300-b6d1-a49776e87e37\") "
Sep 11 13:29:11.414748 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:11.414718    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj" (OuterVolumeSpecName: "kube-api-access-kb5tj") pod "ff008283-df6e-4300-b6d1-a49776e87e37" (UID: "ff008283-df6e-4300-b6d1-a49776e87e37"). InnerVolumeSpecName "kube-api-access-kb5tj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:11.513744 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:11.513686    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kb5tj\" (UniqueName: \"kubernetes.io/projected/ff008283-df6e-4300-b6d1-a49776e87e37-kube-api-access-kb5tj\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:11.972327 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:11.972295    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" path="/var/lib/kubelet/pods/ff008283-df6e-4300-b6d1-a49776e87e37/volumes"
Sep 11 13:29:12.153364 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:12.153324    2573 generic.go:356] "Generic (PLEG): container finished" podID="ff008283-df6e-4300-b6d1-a49776e87e37" containerID="10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199" exitCode=137
Sep 11 13:29:12.153364 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:12.153361    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-d785ad"
Sep 11 13:29:12.153551 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:12.153417    2573 scope.go:117] "RemoveContainer" containerID="10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199"
Sep 11 13:29:12.162111 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:12.162094    2573 scope.go:117] "RemoveContainer" containerID="10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199"
Sep 11 13:29:12.162371 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:12.162353    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199\": container with ID starting with 10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199 not found: ID does not exist" containerID="10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199"
Sep 11 13:29:12.162417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:12.162380    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199"} err="failed to get container status \"10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199\": rpc error: code = NotFound desc = could not find container \"10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199\": container with ID starting with 10f05b791a36c3d23c842dceff52e40c9ac1c5c487fa45cb10e1ac23fdba8199 not found: ID does not exist"
Sep 11 13:29:15.078107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.078066    2573 patch_prober.go:28] interesting pod/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body=
Sep 11 13:29:15.078467 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.078126    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 503"
Sep 11 13:29:15.817207 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.817179    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:29:15.849802 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849733    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.849927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849814    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.849927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849840    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.849927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849880    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.849927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849905    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.850172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849942    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5jh4s\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.850172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.849982    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.850172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850012    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.850172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850083    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token\") pod \"3d81b8dc-6de1-4977-b069-fc711009d86d\" (UID: \"3d81b8dc-6de1-4977-b069-fc711009d86d\") "
Sep 11 13:29:15.850172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850106    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:29:15.850433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850397    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d81b8dc-6de1-4977-b069-fc711009d86d-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.850490 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850466    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:15.850763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850598    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:15.850763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850725    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data" (OuterVolumeSpecName: "istio-data") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:15.850763 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.850740    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:15.853125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.853098    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:29:15.853239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.853151    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token" (OuterVolumeSpecName: "istio-token") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:15.855011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.854984    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s" (OuterVolumeSpecName: "kube-api-access-5jh4s") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "kube-api-access-5jh4s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:15.855147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.855124    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "3d81b8dc-6de1-4977-b069-fc711009d86d" (UID: "3d81b8dc-6de1-4977-b069-fc711009d86d"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:15.951619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951596    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951617    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951626    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951634    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951643    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951652    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5jh4s\" (UniqueName: \"kubernetes.io/projected/3d81b8dc-6de1-4977-b069-fc711009d86d-kube-api-access-5jh4s\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951659    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:15.951791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:15.951667    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d81b8dc-6de1-4977-b069-fc711009d86d-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:16.178948 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.178863    2573 generic.go:356] "Generic (PLEG): container finished" podID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerID="f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888" exitCode=0
Sep 11 13:29:16.178948 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.178913    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"
Sep 11 13:29:16.179397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.178951    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerDied","Data":"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"}
Sep 11 13:29:16.179397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.179009    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5" event={"ID":"3d81b8dc-6de1-4977-b069-fc711009d86d","Type":"ContainerDied","Data":"f0ba57c7e8bd57a7fc428aa14e9a8fcefe368352c7e31445414acda46891bc00"}
Sep 11 13:29:16.179397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.179036    2573 scope.go:117] "RemoveContainer" containerID="f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"
Sep 11 13:29:16.188883 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.188787    2573 scope.go:117] "RemoveContainer" containerID="09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"
Sep 11 13:29:16.197448 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.197427    2573 scope.go:117] "RemoveContainer" containerID="f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"
Sep 11 13:29:16.197785 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:16.197766    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888\": container with ID starting with f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888 not found: ID does not exist" containerID="f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"
Sep 11 13:29:16.197864 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.197793    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888"} err="failed to get container status \"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888\": rpc error: code = NotFound desc = could not find container \"f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888\": container with ID starting with f9a1bcdde92d9bdeba1518a740b23d05661e857b150d4ebe71f7848d9c019888 not found: ID does not exist"
Sep 11 13:29:16.197864 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.197811    2573 scope.go:117] "RemoveContainer" containerID="09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"
Sep 11 13:29:16.198046 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:16.198029    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656\": container with ID starting with 09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656 not found: ID does not exist" containerID="09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"
Sep 11 13:29:16.198089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.198053    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656"} err="failed to get container status \"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656\": rpc error: code = NotFound desc = could not find container \"09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656\": container with ID starting with 09c86e79d82ceaff4b1fb2239505b145fd7df0d6b9db6fcf0bc2e08e7f463656 not found: ID does not exist"
Sep 11 13:29:16.212953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.212929    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:29:16.218749 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:16.218726    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-mt-3d5dcb7b-openshift-default-677f4c7b88-tkqt5"]
Sep 11 13:29:17.974349 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:17.974310    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" path="/var/lib/kubelet/pods/3d81b8dc-6de1-4977-b069-fc711009d86d/volumes"
Sep 11 13:29:23.833448 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:23.833407    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:29:25.478030 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.477977    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" podUID="df17393e-15ad-44a7-bba2-90f5b242da7f" containerName="main" containerID="cri-o://b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a" gracePeriod=45
Sep 11 13:29:25.741007 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.740982    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:29:25.836793 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836767    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jlxlg\" (UniqueName: \"kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.836943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836803    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.836943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836865    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.836943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836886    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.836943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836915    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.836943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.836944    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir\") pod \"df17393e-15ad-44a7-bba2-90f5b242da7f\" (UID: \"df17393e-15ad-44a7-bba2-90f5b242da7f\") "
Sep 11 13:29:25.837230 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.837143    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache" (OuterVolumeSpecName: "model-cache") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:25.837230 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.837163    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home" (OuterVolumeSpecName: "home") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:25.837332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.837260    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:25.839007 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.838983    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm" (OuterVolumeSpecName: "dshm") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:29:25.839123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.839041    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:29:25.839123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.839047    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg" (OuterVolumeSpecName: "kube-api-access-jlxlg") pod "df17393e-15ad-44a7-bba2-90f5b242da7f" (UID: "df17393e-15ad-44a7-bba2-90f5b242da7f"). InnerVolumeSpecName "kube-api-access-jlxlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:29:25.938002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.937981    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.938002 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.938001    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.938128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.938010    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/df17393e-15ad-44a7-bba2-90f5b242da7f-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.938128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.938017    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.938128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.938025    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jlxlg\" (UniqueName: \"kubernetes.io/projected/df17393e-15ad-44a7-bba2-90f5b242da7f-kube-api-access-jlxlg\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:25.938128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:25.938032    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/df17393e-15ad-44a7-bba2-90f5b242da7f-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:29:26.244397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.244362    2573 generic.go:356] "Generic (PLEG): container finished" podID="df17393e-15ad-44a7-bba2-90f5b242da7f" containerID="b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a" exitCode=0
Sep 11 13:29:26.244554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.244412    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"
Sep 11 13:29:26.244554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.244416    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" event={"ID":"df17393e-15ad-44a7-bba2-90f5b242da7f","Type":"ContainerDied","Data":"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"}
Sep 11 13:29:26.244554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.244524    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc" event={"ID":"df17393e-15ad-44a7-bba2-90f5b242da7f","Type":"ContainerDied","Data":"92cec6647f40377317ffb992d160c160be959fb16edb19b5c008c4ed15172504"}
Sep 11 13:29:26.244554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.244543    2573 scope.go:117] "RemoveContainer" containerID="b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"
Sep 11 13:29:26.254533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.254517    2573 scope.go:117] "RemoveContainer" containerID="b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"
Sep 11 13:29:26.254861 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:29:26.254843    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a\": container with ID starting with b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a not found: ID does not exist" containerID="b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"
Sep 11 13:29:26.254934 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.254868    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a"} err="failed to get container status \"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a\": rpc error: code = NotFound desc = could not find container \"b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a\": container with ID starting with b029e8e0d6230b8c0e4cfd2c16991a42258fe841fa4d79fba2d4246d416dfc7a not found: ID does not exist"
Sep 11 13:29:26.278130 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.278107    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:29:26.287429 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:26.287405    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/auth-test-model-12f386-kserve-6b67d9d6b7-stxbc"]
Sep 11 13:29:27.971851 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:27.971810    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="df17393e-15ad-44a7-bba2-90f5b242da7f" path="/var/lib/kubelet/pods/df17393e-15ad-44a7-bba2-90f5b242da7f/volumes"
Sep 11 13:29:41.455227 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455192    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:29:41.455594 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455581    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.455594 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455595    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455608    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" containerName="test-curl-7696-d785ad"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455614    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" containerName="test-curl-7696-d785ad"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455621    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="df17393e-15ad-44a7-bba2-90f5b242da7f" containerName="main"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455626    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="df17393e-15ad-44a7-bba2-90f5b242da7f" containerName="main"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455640    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.455673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455644    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.455875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455701    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.455875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455727    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ff008283-df6e-4300-b6d1-a49776e87e37" containerName="test-curl-7696-d785ad"
Sep 11 13:29:41.455875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455736    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="df17393e-15ad-44a7-bba2-90f5b242da7f" containerName="main"
Sep 11 13:29:41.455875 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.455742    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3d81b8dc-6de1-4977-b069-fc711009d86d" containerName="istio-proxy"
Sep 11 13:29:41.459933 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.459918    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.466951 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.466932    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"openshift-service-ca.crt\""
Sep 11 13:29:41.467095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.466974    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"sub-test-model-6ffb6a-kserve-self-signed-certs\""
Sep 11 13:29:41.467095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.467004    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"default-dockercfg-bxnq5\""
Sep 11 13:29:41.467240 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.467227    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"kube-root-ca.crt\""
Sep 11 13:29:41.476389 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.476365    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:29:41.560979 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.560946    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.561124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.560995    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.561124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.561021    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qnpmj\" (UniqueName: \"kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.561124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.561069    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.561124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.561120    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.561367 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.561152    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.661805 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661773    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.661952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661825    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.661952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661854    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.661952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661897    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.661952 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661926    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.662167 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.661950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qnpmj\" (UniqueName: \"kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.662225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.662173    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.662282 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.662234    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.662282 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.662260    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.664152 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.664124    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.664439 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.664420    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.670932 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.670914    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qnpmj\" (UniqueName: \"kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj\") pod \"sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:41.768587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:41.768568    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:42.056768 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:42.056744    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:29:42.058564 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:29:42.058532    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd5a3e366_1a26_4cae_9927_83549b5af1f0.slice/crio-6a41c3524dc17bbf0476d799167b312bc9779b66b35b0a1eecefd58741b32f05 WatchSource:0}: Error finding container 6a41c3524dc17bbf0476d799167b312bc9779b66b35b0a1eecefd58741b32f05: Status 404 returned error can't find the container with id 6a41c3524dc17bbf0476d799167b312bc9779b66b35b0a1eecefd58741b32f05
Sep 11 13:29:42.336515 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:42.336414    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" event={"ID":"d5a3e366-1a26-4cae-9927-83549b5af1f0","Type":"ContainerStarted","Data":"3d22e200623b97d837109afcce8341d01525ace8f739791d665c80e268de04cd"}
Sep 11 13:29:42.336515 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:42.336455    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" event={"ID":"d5a3e366-1a26-4cae-9927-83549b5af1f0","Type":"ContainerStarted","Data":"6a41c3524dc17bbf0476d799167b312bc9779b66b35b0a1eecefd58741b32f05"}
Sep 11 13:29:42.364451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:42.364346    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" podStartSLOduration=1.364328445 podStartE2EDuration="1.364328445s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:29:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:29:42.363562124 +0000 UTC m=+1496.883599470" watchObservedRunningTime="2026-09-11 13:29:42.364328445 +0000 UTC m=+1496.884365791"
Sep 11 13:29:45.652955 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:45.652912    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:29:46.101837 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:46.101807    2573 scope.go:117] "RemoveContainer" containerID="0459501966aa89442333e0381f42f3cfd05bf80e733b6e344de8a81236ffe898"
Sep 11 13:29:51.769218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:51.769175    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:51.769218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:51.769226    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:51.781611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:51.781582    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:29:52.399836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:29:52.399810    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:30:00.163403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.163367    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"]
Sep 11 13:30:00.166860 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.166840    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"
Sep 11 13:30:00.172254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.172238    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-k6rzc\""
Sep 11 13:30:00.176012 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.175997    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"
Sep 11 13:30:00.192224 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.192202    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"]
Sep 11 13:30:00.453267 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:00.453243    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"]
Sep 11 13:30:00.455145 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:30:00.455116    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5877c76_efcd_4005_9e50_7b059d76f4bf.slice/crio-734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce WatchSource:0}: Error finding container 734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce: Status 404 returned error can't find the container with id 734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce
Sep 11 13:30:01.435177 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:01.435144    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b" event={"ID":"d5877c76-efcd-4005-9e50-7b059d76f4bf","Type":"ContainerStarted","Data":"734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce"}
Sep 11 13:30:15.915758 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.915659    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:30:15.919213 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.919196    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:15.922651 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.922626    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-a-0326b6\""
Sep 11 13:30:15.922791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.922628    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-a-0326b6-dockercfg-rs8rj\""
Sep 11 13:30:15.934843 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.934820    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:30:15.953485 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.953459    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2959\" (UniqueName: \"kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:15.953627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.953519    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:15.963904 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.963883    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:30:15.967940 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.967923    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:15.980043 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:15.980024    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:30:16.054218 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.054177    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zs5rk\" (UniqueName: \"kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.054385 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.054225    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2959\" (UniqueName: \"kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:16.054385 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.054256    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:16.054385 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.054338    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.054969 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.054953    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:16.069421 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.069397    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2959\" (UniqueName: \"kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959\") pod \"payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") " pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:16.155480 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.155447    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zs5rk\" (UniqueName: \"kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.155614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.155511    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.156186 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.156167    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.164908 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.164882    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zs5rk\" (UniqueName: \"kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk\") pod \"payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") " pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.229373 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.229314    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:16.278842 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.278811    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:16.519115 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.519086    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:30:16.520775 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:30:16.520750    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3528920c_4834_4953_89dd_f543d01c7eb5.slice/crio-dbe92145d1640e00866df2f0201b2bf346f8c97571828a9befb2d53d74368610 WatchSource:0}: Error finding container dbe92145d1640e00866df2f0201b2bf346f8c97571828a9befb2d53d74368610: Status 404 returned error can't find the container with id dbe92145d1640e00866df2f0201b2bf346f8c97571828a9befb2d53d74368610
Sep 11 13:30:16.574271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:16.574186    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:30:16.575928 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:30:16.575700    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod623eb641_f603_4fef_b5f6_f43d4b09e792.slice/crio-727aabf7e446a8f14bef503364cee687911bdb8a54caf2eb52690b0549fd7419 WatchSource:0}: Error finding container 727aabf7e446a8f14bef503364cee687911bdb8a54caf2eb52690b0549fd7419: Status 404 returned error can't find the container with id 727aabf7e446a8f14bef503364cee687911bdb8a54caf2eb52690b0549fd7419
Sep 11 13:30:17.528921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.528862    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" event={"ID":"3528920c-4834-4953-89dd-f543d01c7eb5","Type":"ContainerStarted","Data":"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"}
Sep 11 13:30:17.529387 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.528932    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:17.529387 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.528964    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" event={"ID":"3528920c-4834-4953-89dd-f543d01c7eb5","Type":"ContainerStarted","Data":"dbe92145d1640e00866df2f0201b2bf346f8c97571828a9befb2d53d74368610"}
Sep 11 13:30:17.530817 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.530789    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" event={"ID":"623eb641-f603-4fef-b5f6-f43d4b09e792","Type":"ContainerStarted","Data":"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"}
Sep 11 13:30:17.530953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.530848    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:17.530953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.530872    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" event={"ID":"623eb641-f603-4fef-b5f6-f43d4b09e792","Type":"ContainerStarted","Data":"727aabf7e446a8f14bef503364cee687911bdb8a54caf2eb52690b0549fd7419"}
Sep 11 13:30:17.555723 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.555398    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" podStartSLOduration=2.555381116 podStartE2EDuration="2.555381116s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:17.555016 +0000 UTC m=+1532.075053346" watchObservedRunningTime="2026-09-11 13:30:17.555381116 +0000 UTC m=+1532.075418462"
Sep 11 13:30:17.585523 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:17.585472    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" podStartSLOduration=2.585451997 podStartE2EDuration="2.585451997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:17.582638677 +0000 UTC m=+1532.102676048" watchObservedRunningTime="2026-09-11 13:30:17.585451997 +0000 UTC m=+1532.105489346"
Sep 11 13:30:19.542864 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:19.542830    2573 generic.go:356] "Generic (PLEG): container finished" podID="d5877c76-efcd-4005-9e50-7b059d76f4bf" containerID="8f5b6e1a09382b2aafd10c1b34a9044a3b1bda03b534263c94702e5ab7569859" exitCode=0
Sep 11 13:30:19.543194 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:19.542912    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b" event={"ID":"d5877c76-efcd-4005-9e50-7b059d76f4bf","Type":"ContainerDied","Data":"8f5b6e1a09382b2aafd10c1b34a9044a3b1bda03b534263c94702e5ab7569859"}
Sep 11 13:30:20.680637 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:20.680614    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"
Sep 11 13:30:21.554488 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:21.554453    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b" event={"ID":"d5877c76-efcd-4005-9e50-7b059d76f4bf","Type":"ContainerDied","Data":"734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce"}
Sep 11 13:30:21.554745 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:21.554494    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"
Sep 11 13:30:21.554745 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:21.554501    2573 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="734faefdc055614cb7b08e5236c1f873dc3f4e8f10f19f490ed11d4e71f4c4ce"
Sep 11 13:30:21.724856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:21.724820    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:30:28.537226 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:28.537185    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:30:28.537602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:28.537262    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:30:51.391856 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.391814    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:30:51.392397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.392377    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5877c76-efcd-4005-9e50-7b059d76f4bf" containerName="cleanup"
Sep 11 13:30:51.392493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.392399    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5877c76-efcd-4005-9e50-7b059d76f4bf" containerName="cleanup"
Sep 11 13:30:51.392547 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.392523    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5877c76-efcd-4005-9e50-7b059d76f4bf" containerName="cleanup"
Sep 11 13:30:51.396968 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.396946    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.399493 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.399471    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-api-b-6c3a36-dockercfg-bmc7w\""
Sep 11 13:30:51.399603 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.399495    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-api-b-6c3a36\""
Sep 11 13:30:51.408585 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.408557    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:30:51.462924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.462899    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:30:51.465268 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.465255    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.483232 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.483207    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:30:51.580389 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.580357    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.580596 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.580428    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqzvc\" (UniqueName: \"kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.580596 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.580536    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.580694 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.580604    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b7lhg\" (UniqueName: \"kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.681956 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.681872    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqzvc\" (UniqueName: \"kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.681956 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.681935    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.682184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.681982    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b7lhg\" (UniqueName: \"kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.682184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.682029    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.682614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.682589    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.682754 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.682733    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.690672 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.690650    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqzvc\" (UniqueName: \"kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc\") pod \"payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") " pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.690788 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.690756    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b7lhg\" (UniqueName: \"kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg\") pod \"payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") " pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.707176 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.707158    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:51.774433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.774365    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:51.980338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:51.980307    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:30:51.982809 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:30:51.982782    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod54fd7fc0_c3dc_4d8d_bfa8_aeadd57f0d0b.slice/crio-e6fdce4e1ff194985df7ad19c5d8c88b38326250c6ffbcbcf3f2e024b2111dc0 WatchSource:0}: Error finding container e6fdce4e1ff194985df7ad19c5d8c88b38326250c6ffbcbcf3f2e024b2111dc0: Status 404 returned error can't find the container with id e6fdce4e1ff194985df7ad19c5d8c88b38326250c6ffbcbcf3f2e024b2111dc0
Sep 11 13:30:52.056898 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.056871    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:30:52.059314 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:30:52.059281    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf86b2810_fd1f_49cc_91ed_c00ed40e8c33.slice/crio-8dc8b9cbbea796c0986b1a908305235a90aa58f0a85736cf510400c417cb63e6 WatchSource:0}: Error finding container 8dc8b9cbbea796c0986b1a908305235a90aa58f0a85736cf510400c417cb63e6: Status 404 returned error can't find the container with id 8dc8b9cbbea796c0986b1a908305235a90aa58f0a85736cf510400c417cb63e6
Sep 11 13:30:52.729004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.728970    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" event={"ID":"f86b2810-fd1f-49cc-91ed-c00ed40e8c33","Type":"ContainerStarted","Data":"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"}
Sep 11 13:30:52.729542 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.729025    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" event={"ID":"f86b2810-fd1f-49cc-91ed-c00ed40e8c33","Type":"ContainerStarted","Data":"8dc8b9cbbea796c0986b1a908305235a90aa58f0a85736cf510400c417cb63e6"}
Sep 11 13:30:52.729542 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.729079    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:30:52.730722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.730684    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" event={"ID":"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b","Type":"ContainerStarted","Data":"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"}
Sep 11 13:30:52.730806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.730747    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:30:52.730806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.730773    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" event={"ID":"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b","Type":"ContainerStarted","Data":"e6fdce4e1ff194985df7ad19c5d8c88b38326250c6ffbcbcf3f2e024b2111dc0"}
Sep 11 13:30:52.753161 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.753110    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" podStartSLOduration=1.753096904 podStartE2EDuration="1.753096904s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:52.752806403 +0000 UTC m=+1567.272843751" watchObservedRunningTime="2026-09-11 13:30:52.753096904 +0000 UTC m=+1567.273134248"
Sep 11 13:30:52.781157 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:52.781110    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" podStartSLOduration=1.781096756 podStartE2EDuration="1.781096756s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:30:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:30:52.779214449 +0000 UTC m=+1567.299251793" watchObservedRunningTime="2026-09-11 13:30:52.781096756 +0000 UTC m=+1567.301134100"
Sep 11 13:30:57.520803 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:30:57.520765    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:31:03.738212 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:03.738178    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:31:03.738643 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:03.738252    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:31:26.722187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:26.722152    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:31:30.326620 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.326587    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-2fe375"]
Sep 11 13:31:30.329325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.329308    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:30.331438 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.331413    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-6zc4w\""
Sep 11 13:31:30.342099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.340473    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-2fe375"]
Sep 11 13:31:30.435146 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.435108    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xv7w7\" (UniqueName: \"kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7\") pod \"test-curl-7696-2fe375\" (UID: \"81005bf2-a10e-4263-bdd8-f92624440641\") " pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:30.536032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.536001    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xv7w7\" (UniqueName: \"kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7\") pod \"test-curl-7696-2fe375\" (UID: \"81005bf2-a10e-4263-bdd8-f92624440641\") " pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:30.545370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.545343    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xv7w7\" (UniqueName: \"kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7\") pod \"test-curl-7696-2fe375\" (UID: \"81005bf2-a10e-4263-bdd8-f92624440641\") " pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:30.646768 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.646665    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:30.923021 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.922998    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-2fe375"]
Sep 11 13:31:30.925178 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:31:30.925146    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod81005bf2_a10e_4263_bdd8_f92624440641.slice/crio-1c21b8579c4875ddce0e1b48fc053160dd5531d0d774ca0e0875b03facc90634 WatchSource:0}: Error finding container 1c21b8579c4875ddce0e1b48fc053160dd5531d0d774ca0e0875b03facc90634: Status 404 returned error can't find the container with id 1c21b8579c4875ddce0e1b48fc053160dd5531d0d774ca0e0875b03facc90634
Sep 11 13:31:30.945664 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:30.945631    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-2fe375" event={"ID":"81005bf2-a10e-4263-bdd8-f92624440641","Type":"ContainerStarted","Data":"1c21b8579c4875ddce0e1b48fc053160dd5531d0d774ca0e0875b03facc90634"}
Sep 11 13:31:31.952438 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:31.952404    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-2fe375" event={"ID":"81005bf2-a10e-4263-bdd8-f92624440641","Type":"ContainerStarted","Data":"ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e"}
Sep 11 13:31:31.970672 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:31.970627    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-2fe375" podStartSLOduration=1.9706127709999999 podStartE2EDuration="1.970612771s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:31:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:31.968143359 +0000 UTC m=+1606.488180714" watchObservedRunningTime="2026-09-11 13:31:31.970612771 +0000 UTC m=+1606.490650116"
Sep 11 13:31:32.547274 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.547239    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-2fe375"]
Sep 11 13:31:32.552797 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.552767    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-2fe375"]
Sep 11 13:31:32.669565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.669492    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/test-curl-7696-2a32a2"]
Sep 11 13:31:32.669958 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.669943    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81005bf2-a10e-4263-bdd8-f92624440641" containerName="test-curl-7696-2fe375"
Sep 11 13:31:32.670016 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.669959    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="81005bf2-a10e-4263-bdd8-f92624440641" containerName="test-curl-7696-2fe375"
Sep 11 13:31:32.670054 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.670019    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="81005bf2-a10e-4263-bdd8-f92624440641" containerName="test-curl-7696-2fe375"
Sep 11 13:31:32.672229 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.672213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:32.678133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.678108    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-2a32a2"]
Sep 11 13:31:32.756839 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.756808    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tz2f4\" (UniqueName: \"kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4\") pod \"test-curl-7696-2a32a2\" (UID: \"37cb1447-a294-4d94-8d8d-4205c1dec6ce\") " pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:32.857697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.857644    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tz2f4\" (UniqueName: \"kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4\") pod \"test-curl-7696-2a32a2\" (UID: \"37cb1447-a294-4d94-8d8d-4205c1dec6ce\") " pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:32.866896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.866867    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tz2f4\" (UniqueName: \"kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4\") pod \"test-curl-7696-2a32a2\" (UID: \"37cb1447-a294-4d94-8d8d-4205c1dec6ce\") " pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:32.957970 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:32.957881    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-2fe375" container="test-curl-7696-2fe375"
Sep 11 13:31:32.983011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:32.982981    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:33.256159 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:33.256131    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/test-curl-7696-2a32a2"]
Sep 11 13:31:33.257870 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:31:33.257839    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod37cb1447_a294_4d94_8d8d_4205c1dec6ce.slice/crio-8fd7d30bfe07a3670888ba1777b763d682e7332946b218ced6fa04b410b6094c WatchSource:0}: Error finding container 8fd7d30bfe07a3670888ba1777b763d682e7332946b218ced6fa04b410b6094c: Status 404 returned error can't find the container with id 8fd7d30bfe07a3670888ba1777b763d682e7332946b218ced6fa04b410b6094c
Sep 11 13:31:33.964389 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:33.964347    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-2a32a2" event={"ID":"37cb1447-a294-4d94-8d8d-4205c1dec6ce","Type":"ContainerStarted","Data":"c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1"}
Sep 11 13:31:33.964836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:33.964403    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/test-curl-7696-2a32a2" event={"ID":"37cb1447-a294-4d94-8d8d-4205c1dec6ce","Type":"ContainerStarted","Data":"8fd7d30bfe07a3670888ba1777b763d682e7332946b218ced6fa04b410b6094c"}
Sep 11 13:31:33.964836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:33.964438    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-2fe375" podUID="81005bf2-a10e-4263-bdd8-f92624440641" containerName="test-curl-7696-2fe375" containerID="cri-o://ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e" gracePeriod=2
Sep 11 13:31:33.980578 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:33.980533    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2fe375\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="81005bf2-a10e-4263-bdd8-f92624440641" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:33.980842 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:33.980802    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/test-curl-7696-2a32a2" podStartSLOduration=1.9807880610000002 podStartE2EDuration="1.980788061s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:31:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:33.978803378 +0000 UTC m=+1608.498840723" watchObservedRunningTime="2026-09-11 13:31:33.980788061 +0000 UTC m=+1608.500825405"
Sep 11 13:31:34.355700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.355663    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:31:34.356409 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.356166    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" podUID="623eb641-f603-4fef-b5f6-f43d4b09e792" containerName="payload-processing" containerID="cri-o://42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1" gracePeriod=30
Sep 11 13:31:34.361015 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.360990    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:31:34.361212 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.361185    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" podUID="3528920c-4834-4953-89dd-f543d01c7eb5" containerName="payload-pre-processing" containerID="cri-o://bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068" gracePeriod=30
Sep 11 13:31:34.581069 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.581036    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/test-curl-7696-2a32a2"]
Sep 11 13:31:34.587020 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.586984    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/test-curl-7696-2a32a2"]
Sep 11 13:31:34.634780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.634758    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:31:34.653243 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.653212    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:31:34.777236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777202    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume\") pod \"623eb641-f603-4fef-b5f6-f43d4b09e792\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") "
Sep 11 13:31:34.777417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777254    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zs5rk\" (UniqueName: \"kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk\") pod \"623eb641-f603-4fef-b5f6-f43d4b09e792\" (UID: \"623eb641-f603-4fef-b5f6-f43d4b09e792\") "
Sep 11 13:31:34.777417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777286    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2959\" (UniqueName: \"kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959\") pod \"3528920c-4834-4953-89dd-f543d01c7eb5\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") "
Sep 11 13:31:34.777417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777340    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume\") pod \"3528920c-4834-4953-89dd-f543d01c7eb5\" (UID: \"3528920c-4834-4953-89dd-f543d01c7eb5\") "
Sep 11 13:31:34.777627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777596    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "623eb641-f603-4fef-b5f6-f43d4b09e792" (UID: "623eb641-f603-4fef-b5f6-f43d4b09e792"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:34.777837 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.777811    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "3528920c-4834-4953-89dd-f543d01c7eb5" (UID: "3528920c-4834-4953-89dd-f543d01c7eb5"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:34.779379 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.779358    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959" (OuterVolumeSpecName: "kube-api-access-f2959") pod "3528920c-4834-4953-89dd-f543d01c7eb5" (UID: "3528920c-4834-4953-89dd-f543d01c7eb5"). InnerVolumeSpecName "kube-api-access-f2959". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:34.779454 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.779396    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk" (OuterVolumeSpecName: "kube-api-access-zs5rk") pod "623eb641-f603-4fef-b5f6-f43d4b09e792" (UID: "623eb641-f603-4fef-b5f6-f43d4b09e792"). InnerVolumeSpecName "kube-api-access-zs5rk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:34.878395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.878313    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/3528920c-4834-4953-89dd-f543d01c7eb5-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.878395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.878345    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/623eb641-f603-4fef-b5f6-f43d4b09e792-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.878395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.878358    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zs5rk\" (UniqueName: \"kubernetes.io/projected/623eb641-f603-4fef-b5f6-f43d4b09e792-kube-api-access-zs5rk\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.878395 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.878369    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2959\" (UniqueName: \"kubernetes.io/projected/3528920c-4834-4953-89dd-f543d01c7eb5-kube-api-access-f2959\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:34.971700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.971669    2573 generic.go:356] "Generic (PLEG): container finished" podID="3528920c-4834-4953-89dd-f543d01c7eb5" containerID="bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068" exitCode=0
Sep 11 13:31:34.972094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.971720    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"
Sep 11 13:31:34.972094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.971737    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" event={"ID":"3528920c-4834-4953-89dd-f543d01c7eb5","Type":"ContainerDied","Data":"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"}
Sep 11 13:31:34.972094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.971786    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47" event={"ID":"3528920c-4834-4953-89dd-f543d01c7eb5","Type":"ContainerDied","Data":"dbe92145d1640e00866df2f0201b2bf346f8c97571828a9befb2d53d74368610"}
Sep 11 13:31:34.972094 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.971809    2573 scope.go:117] "RemoveContainer" containerID="bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"
Sep 11 13:31:34.973477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.973456    2573 generic.go:356] "Generic (PLEG): container finished" podID="623eb641-f603-4fef-b5f6-f43d4b09e792" containerID="42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1" exitCode=0
Sep 11 13:31:34.973601 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.973506    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"
Sep 11 13:31:34.973665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.973599    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" event={"ID":"623eb641-f603-4fef-b5f6-f43d4b09e792","Type":"ContainerDied","Data":"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"}
Sep 11 13:31:34.973665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.973655    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz" event={"ID":"623eb641-f603-4fef-b5f6-f43d4b09e792","Type":"ContainerDied","Data":"727aabf7e446a8f14bef503364cee687911bdb8a54caf2eb52690b0549fd7419"}
Sep 11 13:31:34.973895 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:34.973877    2573 kuberuntime_manager.go:649] "Missing actuated resource record" pod="opendatahub/test-curl-7696-2a32a2" container="test-curl-7696-2a32a2"
Sep 11 13:31:34.983060 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.983038    2573 scope.go:117] "RemoveContainer" containerID="bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"
Sep 11 13:31:34.983349 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:34.983331    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068\": container with ID starting with bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068 not found: ID does not exist" containerID="bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"
Sep 11 13:31:34.983410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.983357    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068"} err="failed to get container status \"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068\": rpc error: code = NotFound desc = could not find container \"bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068\": container with ID starting with bf476b703113388f74aaab4a9094884fabf552414b6550e1061369ac21806068 not found: ID does not exist"
Sep 11 13:31:34.983410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.983372    2573 scope.go:117] "RemoveContainer" containerID="42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"
Sep 11 13:31:34.991808 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.991790    2573 scope.go:117] "RemoveContainer" containerID="42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"
Sep 11 13:31:34.992060 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:34.992040    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1\": container with ID starting with 42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1 not found: ID does not exist" containerID="42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"
Sep 11 13:31:34.992113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.992065    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1"} err="failed to get container status \"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1\": rpc error: code = NotFound desc = could not find container \"42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1\": container with ID starting with 42407e0a6f0f7a9268b19d213a1c57045b221e478f7c0337b2d7a108f77235a1 not found: ID does not exist"
Sep 11 13:31:34.998741 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:34.998695    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:31:35.003566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.003547    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-a-0326b6-797b65fcc8-rmh47"]
Sep 11 13:31:35.017799 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.017776    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:31:35.024689 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.024667    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-a-0326b6-7fcd55fd8f-2t8kz"]
Sep 11 13:31:35.975326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.975291    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3528920c-4834-4953-89dd-f543d01c7eb5" path="/var/lib/kubelet/pods/3528920c-4834-4953-89dd-f543d01c7eb5/volumes"
Sep 11 13:31:35.975902 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.975881    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="623eb641-f603-4fef-b5f6-f43d4b09e792" path="/var/lib/kubelet/pods/623eb641-f603-4fef-b5f6-f43d4b09e792/volumes"
Sep 11 13:31:35.981052 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:35.981021    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/test-curl-7696-2a32a2" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" containerName="test-curl-7696-2a32a2" containerID="cri-o://c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1" gracePeriod=2
Sep 11 13:31:36.145384 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.145362    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:36.147325 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:36.147301    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2fe375\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="81005bf2-a10e-4263-bdd8-f92624440641" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:36.291273 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.291244    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xv7w7\" (UniqueName: \"kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7\") pod \"81005bf2-a10e-4263-bdd8-f92624440641\" (UID: \"81005bf2-a10e-4263-bdd8-f92624440641\") "
Sep 11 13:31:36.293504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.293476    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7" (OuterVolumeSpecName: "kube-api-access-xv7w7") pod "81005bf2-a10e-4263-bdd8-f92624440641" (UID: "81005bf2-a10e-4263-bdd8-f92624440641"). InnerVolumeSpecName "kube-api-access-xv7w7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:36.392587 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.392556    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xv7w7\" (UniqueName: \"kubernetes.io/projected/81005bf2-a10e-4263-bdd8-f92624440641-kube-api-access-xv7w7\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:36.861030 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.860997    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:31:36.863821 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:36.863782    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2fe375\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="81005bf2-a10e-4263-bdd8-f92624440641" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:36.988206 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.988164    2573 generic.go:356] "Generic (PLEG): container finished" podID="81005bf2-a10e-4263-bdd8-f92624440641" containerID="ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e" exitCode=137
Sep 11 13:31:36.988637 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.988223    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:36.988637 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:36.988227    2573 scope.go:117] "RemoveContainer" containerID="ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e"
Sep 11 13:31:36.990330 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:36.990303    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2fe375\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="81005bf2-a10e-4263-bdd8-f92624440641" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:37.001140 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:37.001100    2573 scope.go:117] "RemoveContainer" containerID="ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e"
Sep 11 13:31:37.001518 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:37.001491    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e\": container with ID starting with ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e not found: ID does not exist" containerID="ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e"
Sep 11 13:31:37.001599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:37.001530    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e"} err="failed to get container status \"ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e\": rpc error: code = NotFound desc = could not find container \"ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e\": container with ID starting with ab8d5e1a52e00b6e82bf1e57a7c96e3552749a83e90256bbe4db855658f5c05e not found: ID does not exist"
Sep 11 13:31:37.006498 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:37.006472    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2fe375\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="81005bf2-a10e-4263-bdd8-f92624440641" pod="opendatahub/test-curl-7696-2fe375"
Sep 11 13:31:37.972491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:37.972461    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81005bf2-a10e-4263-bdd8-f92624440641" path="/var/lib/kubelet/pods/81005bf2-a10e-4263-bdd8-f92624440641/volumes"
Sep 11 13:31:38.174090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:38.174067    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:38.314360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:38.314333    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tz2f4\" (UniqueName: \"kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4\") pod \"37cb1447-a294-4d94-8d8d-4205c1dec6ce\" (UID: \"37cb1447-a294-4d94-8d8d-4205c1dec6ce\") "
Sep 11 13:31:38.316349 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:38.316320    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4" (OuterVolumeSpecName: "kube-api-access-tz2f4") pod "37cb1447-a294-4d94-8d8d-4205c1dec6ce" (UID: "37cb1447-a294-4d94-8d8d-4205c1dec6ce"). InnerVolumeSpecName "kube-api-access-tz2f4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:38.415327 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:38.415302    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tz2f4\" (UniqueName: \"kubernetes.io/projected/37cb1447-a294-4d94-8d8d-4205c1dec6ce-kube-api-access-tz2f4\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:39.010062 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.010028    2573 generic.go:356] "Generic (PLEG): container finished" podID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" containerID="c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1" exitCode=137
Sep 11 13:31:39.010242 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.010069    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:39.010242 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.010095    2573 scope.go:117] "RemoveContainer" containerID="c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1"
Sep 11 13:31:39.019089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.019071    2573 scope.go:117] "RemoveContainer" containerID="c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1"
Sep 11 13:31:39.019343 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:39.019325    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1\": container with ID starting with c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1 not found: ID does not exist" containerID="c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1"
Sep 11 13:31:39.019399 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.019350    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1"} err="failed to get container status \"c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1\": rpc error: code = NotFound desc = could not find container \"c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1\": container with ID starting with c88ee3a634626c41fa1ac9a8eb2f732c55c41bef04d0da38d6ae05e8e9ec74d1 not found: ID does not exist"
Sep 11 13:31:39.019988 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:39.019955    2573 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7696-2a32a2\" is forbidden: User \"system:node:ip-10-0-142-204.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"opendatahub\": no relationship found between node 'ip-10-0-142-204.ec2.internal' and this object" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" pod="opendatahub/test-curl-7696-2a32a2"
Sep 11 13:31:39.973397 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:39.973361    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" path="/var/lib/kubelet/pods/37cb1447-a294-4d94-8d8d-4205c1dec6ce/volumes"
Sep 11 13:31:42.229433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:42.229401    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:31:48.023031 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.022996    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023365    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="623eb641-f603-4fef-b5f6-f43d4b09e792" containerName="payload-processing"
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023376    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="623eb641-f603-4fef-b5f6-f43d4b09e792" containerName="payload-processing"
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023400    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3528920c-4834-4953-89dd-f543d01c7eb5" containerName="payload-pre-processing"
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023405    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="3528920c-4834-4953-89dd-f543d01c7eb5" containerName="payload-pre-processing"
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023416    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" containerName="test-curl-7696-2a32a2"
Sep 11 13:31:48.023455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023421    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" containerName="test-curl-7696-2a32a2"
Sep 11 13:31:48.023669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023478    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="623eb641-f603-4fef-b5f6-f43d4b09e792" containerName="payload-processing"
Sep 11 13:31:48.023669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023484    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="37cb1447-a294-4d94-8d8d-4205c1dec6ce" containerName="test-curl-7696-2a32a2"
Sep 11 13:31:48.023669 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.023493    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="3528920c-4834-4953-89dd-f543d01c7eb5" containerName="payload-pre-processing"
Sep 11 13:31:48.031602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.031578    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.034024 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.034003    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-w3-6ffb6a\"/\"rate-test-model-6ffb6a-kserve-self-signed-certs\""
Sep 11 13:31:48.038535 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.038509    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:31:48.098070 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098041    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.098070 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098073    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.098294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098094    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.098294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098157    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.098294 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098286    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l5z7f\" (UniqueName: \"kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.098482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.098355    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199443    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l5z7f\" (UniqueName: \"kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199498    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199527    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199544    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199563    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199663 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199585    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.199978 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199923    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.200040 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.199996    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.200087 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.200064    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.201996 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.201940    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.202113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.202096    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.210534 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.210507    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l5z7f\" (UniqueName: \"kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f\") pod \"rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") " pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.342796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.342724    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:48.618566 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:48.618542    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:31:48.620564 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:31:48.620539    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod32264fbb_bbff_4754_963e_e0eb2db8e37b.slice/crio-ad927168553d90810ecc68a0400686362cfc1a18bcdad69d688b37eb72a28c99 WatchSource:0}: Error finding container ad927168553d90810ecc68a0400686362cfc1a18bcdad69d688b37eb72a28c99: Status 404 returned error can't find the container with id ad927168553d90810ecc68a0400686362cfc1a18bcdad69d688b37eb72a28c99
Sep 11 13:31:49.066004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:49.065963    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" event={"ID":"32264fbb-bbff-4754-963e-e0eb2db8e37b","Type":"ContainerStarted","Data":"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"}
Sep 11 13:31:49.066383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:49.066011    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" event={"ID":"32264fbb-bbff-4754-963e-e0eb2db8e37b","Type":"ContainerStarted","Data":"ad927168553d90810ecc68a0400686362cfc1a18bcdad69d688b37eb72a28c99"}
Sep 11 13:31:49.089315 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:49.089265    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" podStartSLOduration=2.089250201 podStartE2EDuration="2.089250201s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:31:47 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:31:49.086876749 +0000 UTC m=+1623.606914095" watchObservedRunningTime="2026-09-11 13:31:49.089250201 +0000 UTC m=+1623.609287561"
Sep 11 13:31:51.880295 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:51.880259    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" podUID="d5a3e366-1a26-4cae-9927-83549b5af1f0" containerName="main" containerID="cri-o://3d22e200623b97d837109afcce8341d01525ace8f739791d665c80e268de04cd" gracePeriod=45
Sep 11 13:31:52.085673 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.085643    2573 generic.go:356] "Generic (PLEG): container finished" podID="d5a3e366-1a26-4cae-9927-83549b5af1f0" containerID="3d22e200623b97d837109afcce8341d01525ace8f739791d665c80e268de04cd" exitCode=0
Sep 11 13:31:52.085840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.085693    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" event={"ID":"d5a3e366-1a26-4cae-9927-83549b5af1f0","Type":"ContainerDied","Data":"3d22e200623b97d837109afcce8341d01525ace8f739791d665c80e268de04cd"}
Sep 11 13:31:52.138908 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.138839    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:31:52.235386 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235361    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235411    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235451    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235472    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235503    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235569    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qnpmj\" (UniqueName: \"kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj\") pod \"d5a3e366-1a26-4cae-9927-83549b5af1f0\" (UID: \"d5a3e366-1a26-4cae-9927-83549b5af1f0\") "
Sep 11 13:31:52.235759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235641    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home" (OuterVolumeSpecName: "home") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:52.235759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235742    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:52.235925 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235874    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache" (OuterVolumeSpecName: "model-cache") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:52.235991 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235931    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:52.235991 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.235950    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:52.237519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.237494    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:31:52.237622 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.237536    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm" (OuterVolumeSpecName: "dshm") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:31:52.238013 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.237997    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj" (OuterVolumeSpecName: "kube-api-access-qnpmj") pod "d5a3e366-1a26-4cae-9927-83549b5af1f0" (UID: "d5a3e366-1a26-4cae-9927-83549b5af1f0"). InnerVolumeSpecName "kube-api-access-qnpmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:52.337071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.337048    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:52.337071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.337068    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d5a3e366-1a26-4cae-9927-83549b5af1f0-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:52.337289 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.337079    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qnpmj\" (UniqueName: \"kubernetes.io/projected/d5a3e366-1a26-4cae-9927-83549b5af1f0-kube-api-access-qnpmj\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:52.337289 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:52.337087    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d5a3e366-1a26-4cae-9927-83549b5af1f0-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:53.094058 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.094029    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"
Sep 11 13:31:53.094485 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.094078    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2" event={"ID":"d5a3e366-1a26-4cae-9927-83549b5af1f0","Type":"ContainerDied","Data":"6a41c3524dc17bbf0476d799167b312bc9779b66b35b0a1eecefd58741b32f05"}
Sep 11 13:31:53.094485 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.094137    2573 scope.go:117] "RemoveContainer" containerID="3d22e200623b97d837109afcce8341d01525ace8f739791d665c80e268de04cd"
Sep 11 13:31:53.124082 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.124055    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:31:53.133579 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.133557    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/sub-test-model-6ffb6a-kserve-7c95c7dfb9-dmfk2"]
Sep 11 13:31:53.972298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:53.972268    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5a3e366-1a26-4cae-9927-83549b5af1f0" path="/var/lib/kubelet/pods/d5a3e366-1a26-4cae-9927-83549b5af1f0/volumes"
Sep 11 13:31:57.802431 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:57.802402    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:31:57.802896 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:57.802634    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" podUID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" containerName="payload-processing" containerID="cri-o://b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc" gracePeriod=30
Sep 11 13:31:57.808441 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:57.808411    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:31:57.808671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:57.808642    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" podUID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" containerName="payload-pre-processing" containerID="cri-o://93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea" gracePeriod=30
Sep 11 13:31:58.058801 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.058745    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:31:58.086787 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.086758    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume\") pod \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") "
Sep 11 13:31:58.086936 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.086814    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b7lhg\" (UniqueName: \"kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg\") pod \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\" (UID: \"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b\") "
Sep 11 13:31:58.087165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.087139    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" (UID: "54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:58.089169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.089138    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg" (OuterVolumeSpecName: "kube-api-access-b7lhg") pod "54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" (UID: "54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b"). InnerVolumeSpecName "kube-api-access-b7lhg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:58.091623 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.091602    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:31:58.127731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.125610    2573 generic.go:356] "Generic (PLEG): container finished" podID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" containerID="93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea" exitCode=0
Sep 11 13:31:58.127731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.125683    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"
Sep 11 13:31:58.127731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.125676    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" event={"ID":"f86b2810-fd1f-49cc-91ed-c00ed40e8c33","Type":"ContainerDied","Data":"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"}
Sep 11 13:31:58.127731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.125829    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t" event={"ID":"f86b2810-fd1f-49cc-91ed-c00ed40e8c33","Type":"ContainerDied","Data":"8dc8b9cbbea796c0986b1a908305235a90aa58f0a85736cf510400c417cb63e6"}
Sep 11 13:31:58.127731 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.125869    2573 scope.go:117] "RemoveContainer" containerID="93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.129183    2573 generic.go:356] "Generic (PLEG): container finished" podID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" containerID="b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc" exitCode=0
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.129399    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" event={"ID":"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b","Type":"ContainerDied","Data":"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"}
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.129469    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk" event={"ID":"54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b","Type":"ContainerDied","Data":"e6fdce4e1ff194985df7ad19c5d8c88b38326250c6ffbcbcf3f2e024b2111dc0"}
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.129564    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.141845    2573 scope.go:117] "RemoveContainer" containerID="93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:58.142311    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea\": container with ID starting with 93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea not found: ID does not exist" containerID="93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.142348    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea"} err="failed to get container status \"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea\": rpc error: code = NotFound desc = could not find container \"93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea\": container with ID starting with 93684e3da17be6551cfbbac90dff3c8ab62d53ccae0de40a1a1736ccf187a5ea not found: ID does not exist"
Sep 11 13:31:58.143563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.142373    2573 scope.go:117] "RemoveContainer" containerID="b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"
Sep 11 13:31:58.162576 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.162542    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:31:58.163959 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.163916    2573 scope.go:117] "RemoveContainer" containerID="b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"
Sep 11 13:31:58.164503 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:31:58.164468    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc\": container with ID starting with b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc not found: ID does not exist" containerID="b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"
Sep 11 13:31:58.164628 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.164509    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc"} err="failed to get container status \"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc\": rpc error: code = NotFound desc = could not find container \"b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc\": container with ID starting with b8dd05b6d2c58d5bb1134361f192c1228eff4c0ae7e80500adcadda4369c83fc not found: ID does not exist"
Sep 11 13:31:58.167004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.166982    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-api-b-6c3a36-6f9fcb5549-m2rsk"]
Sep 11 13:31:58.187998 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.187961    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fqzvc\" (UniqueName: \"kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc\") pod \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") "
Sep 11 13:31:58.188169 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.188011    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume\") pod \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\" (UID: \"f86b2810-fd1f-49cc-91ed-c00ed40e8c33\") "
Sep 11 13:31:58.188259 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.188248    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.188298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.188262    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b7lhg\" (UniqueName: \"kubernetes.io/projected/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b-kube-api-access-b7lhg\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.188417 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.188386    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "f86b2810-fd1f-49cc-91ed-c00ed40e8c33" (UID: "f86b2810-fd1f-49cc-91ed-c00ed40e8c33"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:31:58.190114 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.190095    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc" (OuterVolumeSpecName: "kube-api-access-fqzvc") pod "f86b2810-fd1f-49cc-91ed-c00ed40e8c33" (UID: "f86b2810-fd1f-49cc-91ed-c00ed40e8c33"). InnerVolumeSpecName "kube-api-access-fqzvc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:31:58.289469 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.289434    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqzvc\" (UniqueName: \"kubernetes.io/projected/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-kube-api-access-fqzvc\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.289469 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.289462    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/f86b2810-fd1f-49cc-91ed-c00ed40e8c33-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:31:58.343231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.343157    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:58.343231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.343209    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:58.356509 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.356486    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:58.453145 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.453120    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:31:58.462799 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:58.462771    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-api-b-6c3a36-6454c6bdcd-t6p5t"]
Sep 11 13:31:59.148836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:59.148755    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:31:59.972311 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:59.972277    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" path="/var/lib/kubelet/pods/54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b/volumes"
Sep 11 13:31:59.972797 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:31:59.972776    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" path="/var/lib/kubelet/pods/f86b2810-fd1f-49cc-91ed-c00ed40e8c33/volumes"
Sep 11 13:32:02.211620 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:02.211582    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:32:16.174312 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.174278    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"]
Sep 11 13:32:16.181332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.181305    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-shared-a-w2-e0ebd7-29818890-4nf9b"]
Sep 11 13:32:16.219865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.219834    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:32:16.220138 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.220111    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" podUID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" containerName="payload-processing" containerID="cri-o://afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f" gracePeriod=30
Sep 11 13:32:16.236358 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.236333    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:32:16.236634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.236587    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" podUID="ab893501-c190-436a-a225-025b6d0b4d97" containerName="payload-pre-processing" containerID="cri-o://8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557" gracePeriod=30
Sep 11 13:32:16.269302 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:16.269272    2573 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58902ff7_793e_4bc2_95e3_4e4a4d35662c.slice/crio-conmon-afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f.scope\": RecentStats: unable to find data in memory cache]"
Sep 11 13:32:16.522624 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.522602    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:32:16.525857 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.525840    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:32:16.654684 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.654655    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume\") pod \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") "
Sep 11 13:32:16.654865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.654772    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvn8t\" (UniqueName: \"kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t\") pod \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\" (UID: \"58902ff7-793e-4bc2-95e3-4e4a4d35662c\") "
Sep 11 13:32:16.654865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.654818    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gx9ht\" (UniqueName: \"kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht\") pod \"ab893501-c190-436a-a225-025b6d0b4d97\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") "
Sep 11 13:32:16.654865 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.654855    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume\") pod \"ab893501-c190-436a-a225-025b6d0b4d97\" (UID: \"ab893501-c190-436a-a225-025b6d0b4d97\") "
Sep 11 13:32:16.655038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.655015    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "58902ff7-793e-4bc2-95e3-4e4a4d35662c" (UID: "58902ff7-793e-4bc2-95e3-4e4a4d35662c"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:16.655305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.655277    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/58902ff7-793e-4bc2-95e3-4e4a4d35662c-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.655305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.655274    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "ab893501-c190-436a-a225-025b6d0b4d97" (UID: "ab893501-c190-436a-a225-025b6d0b4d97"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:16.656983 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.656956    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t" (OuterVolumeSpecName: "kube-api-access-tvn8t") pod "58902ff7-793e-4bc2-95e3-4e4a4d35662c" (UID: "58902ff7-793e-4bc2-95e3-4e4a4d35662c"). InnerVolumeSpecName "kube-api-access-tvn8t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:16.657089 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.656999    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht" (OuterVolumeSpecName: "kube-api-access-gx9ht") pod "ab893501-c190-436a-a225-025b6d0b4d97" (UID: "ab893501-c190-436a-a225-025b6d0b4d97"). InnerVolumeSpecName "kube-api-access-gx9ht". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:16.756688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.756657    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/ab893501-c190-436a-a225-025b6d0b4d97-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.756688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.756685    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvn8t\" (UniqueName: \"kubernetes.io/projected/58902ff7-793e-4bc2-95e3-4e4a4d35662c-kube-api-access-tvn8t\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:16.756866 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:16.756698    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gx9ht\" (UniqueName: \"kubernetes.io/projected/ab893501-c190-436a-a225-025b6d0b4d97-kube-api-access-gx9ht\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:17.233198 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.233117    2573 generic.go:356] "Generic (PLEG): container finished" podID="ab893501-c190-436a-a225-025b6d0b4d97" containerID="8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557" exitCode=0
Sep 11 13:32:17.233198 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.233163    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"
Sep 11 13:32:17.233671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.233192    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" event={"ID":"ab893501-c190-436a-a225-025b6d0b4d97","Type":"ContainerDied","Data":"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"}
Sep 11 13:32:17.233671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.233254    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2" event={"ID":"ab893501-c190-436a-a225-025b6d0b4d97","Type":"ContainerDied","Data":"0785d11db3579af7d4e1ba44d0c5e36a1578feb0dbd01f0383ca3f5105020322"}
Sep 11 13:32:17.233671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.233278    2573 scope.go:117] "RemoveContainer" containerID="8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"
Sep 11 13:32:17.234967 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.234943    2573 generic.go:356] "Generic (PLEG): container finished" podID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" containerID="afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f" exitCode=0
Sep 11 13:32:17.235071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.234982    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" event={"ID":"58902ff7-793e-4bc2-95e3-4e4a4d35662c","Type":"ContainerDied","Data":"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"}
Sep 11 13:32:17.235071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.234990    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"
Sep 11 13:32:17.235071 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.235013    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h" event={"ID":"58902ff7-793e-4bc2-95e3-4e4a4d35662c","Type":"ContainerDied","Data":"a7ccdc05439c0f3c26d2120561e324b77a46afadef844d8a475696b4907bdbd6"}
Sep 11 13:32:17.244499 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.244469    2573 scope.go:117] "RemoveContainer" containerID="8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"
Sep 11 13:32:17.244777 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:17.244757    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557\": container with ID starting with 8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557 not found: ID does not exist" containerID="8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"
Sep 11 13:32:17.244863 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.244782    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557"} err="failed to get container status \"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557\": rpc error: code = NotFound desc = could not find container \"8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557\": container with ID starting with 8a264e70af24702ed9140857a6b9f4a9431c472c6ba88d9c6a4c4f60618c5557 not found: ID does not exist"
Sep 11 13:32:17.244863 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.244798    2573 scope.go:117] "RemoveContainer" containerID="afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"
Sep 11 13:32:17.252377 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.252358    2573 scope.go:117] "RemoveContainer" containerID="afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"
Sep 11 13:32:17.252629 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:17.252612    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f\": container with ID starting with afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f not found: ID does not exist" containerID="afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"
Sep 11 13:32:17.252677 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.252634    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f"} err="failed to get container status \"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f\": rpc error: code = NotFound desc = could not find container \"afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f\": container with ID starting with afd8c9825fef308cd12c0da7de2fb00e6472b29ffb5c454c13a62cba8014116f not found: ID does not exist"
Sep 11 13:32:17.307829 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.307807    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:32:17.314849 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.314828    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-pre-processing-e2e-shared-a-w2-e0ebd7-979cbfd54-89dm2"]
Sep 11 13:32:17.353781 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.353761    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:32:17.365277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.365252    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-shared-a-w2-e0ebd7-59495c7867-bkc9h"]
Sep 11 13:32:17.972846 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.972802    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" path="/var/lib/kubelet/pods/58902ff7-793e-4bc2-95e3-4e4a4d35662c/volumes"
Sep 11 13:32:17.973252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.973231    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab893501-c190-436a-a225-025b6d0b4d97" path="/var/lib/kubelet/pods/ab893501-c190-436a-a225-025b6d0b4d97/volumes"
Sep 11 13:32:17.973545 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:17.973527    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5877c76-efcd-4005-9e50-7b059d76f4bf" path="/var/lib/kubelet/pods/d5877c76-efcd-4005-9e50-7b059d76f4bf/volumes"
Sep 11 13:32:20.404536 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404495    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404874    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab893501-c190-436a-a225-025b6d0b4d97" containerName="payload-pre-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404890    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab893501-c190-436a-a225-025b6d0b4d97" containerName="payload-pre-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404899    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5a3e366-1a26-4cae-9927-83549b5af1f0" containerName="main"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404904    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5a3e366-1a26-4cae-9927-83549b5af1f0" containerName="main"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404918    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" containerName="payload-pre-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404925    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" containerName="payload-pre-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404932    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" containerName="payload-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404938    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" containerName="payload-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404954    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" containerName="payload-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.404962    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" containerName="payload-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.405024    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="f86b2810-fd1f-49cc-91ed-c00ed40e8c33" containerName="payload-pre-processing"
Sep 11 13:32:20.405032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.405035    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="ab893501-c190-436a-a225-025b6d0b4d97" containerName="payload-pre-processing"
Sep 11 13:32:20.405635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.405045    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="58902ff7-793e-4bc2-95e3-4e4a4d35662c" containerName="payload-processing"
Sep 11 13:32:20.405635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.405051    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d5a3e366-1a26-4cae-9927-83549b5af1f0" containerName="main"
Sep 11 13:32:20.405635 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.405057    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="54fd7fc0-c3dc-4d8d-bfa8-aeadd57f0d0b" containerName="payload-processing"
Sep 11 13:32:20.407986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.407970    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.413636 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.413616    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"default-dockercfg-285dr\""
Sep 11 13:32:20.413636 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.413629    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"kube-root-ca.crt\""
Sep 11 13:32:20.414352 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.414334    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"openshift-service-ca.crt\""
Sep 11 13:32:20.414463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.414428    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-w3-12f386\"/\"rate-test-model-12f386-kserve-self-signed-certs\""
Sep 11 13:32:20.428847 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.428826    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:32:20.436916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.436891    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:32:20.490825 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.490798    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.490954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.490834    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.490954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.490868    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.490954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.490920    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.490954 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.490940    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-whvtd\" (UniqueName: \"kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.491106 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.491011    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592079    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592122    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592176    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592232    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592262    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-whvtd\" (UniqueName: \"kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592527 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592500    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592651 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592601    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.592862 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.592841    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.594547 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.594524    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.594743 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.594723    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.602548 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.602528    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-whvtd\" (UniqueName: \"kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd\") pod \"rate-test-model-12f386-kserve-58f989d4b7-rhcdc\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") " pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.717509 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.717417    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:20.995771 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:20.995744    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:32:20.997690 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:32:20.997667    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7d8fb06_d4c7_4aee_9a4b_faa64816b2c9.slice/crio-ce439442b91b46d47c52a3ee9a56f522b47a5d676acc59bbcd36fe468bec8469 WatchSource:0}: Error finding container ce439442b91b46d47c52a3ee9a56f522b47a5d676acc59bbcd36fe468bec8469: Status 404 returned error can't find the container with id ce439442b91b46d47c52a3ee9a56f522b47a5d676acc59bbcd36fe468bec8469
Sep 11 13:32:21.262123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:21.262039    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" event={"ID":"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9","Type":"ContainerStarted","Data":"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"}
Sep 11 13:32:21.262123 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:21.262083    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" event={"ID":"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9","Type":"ContainerStarted","Data":"ce439442b91b46d47c52a3ee9a56f522b47a5d676acc59bbcd36fe468bec8469"}
Sep 11 13:32:23.858128 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:23.858072    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" podStartSLOduration=3.858048588 podStartE2EDuration="3.858048588s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:32:20 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:32:21.289238516 +0000 UTC m=+1655.809275861" watchObservedRunningTime="2026-09-11 13:32:23.858048588 +0000 UTC m=+1658.378085934"
Sep 11 13:32:23.859838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:23.859817    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:32:23.860088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:23.860065    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy" containerID="cri-o://8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b" gracePeriod=30
Sep 11 13:32:29.104973 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.104949    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:32:29.271206 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271183    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271221    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271254    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271288    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271307    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271351    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271392    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271439    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271463    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmwqx\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx\") pod \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\" (UID: \"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303\") "
Sep 11 13:32:29.271648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271487    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data" (OuterVolumeSpecName: "istio-data") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:29.271648 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271611    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:29.271912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271670    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:29.271912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271771    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:29.271912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271813    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.271912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271834    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.271912 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271848    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.272081 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.271991    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:29.273583 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.273552    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:32:29.273690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.273626    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token" (OuterVolumeSpecName: "istio-token") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:29.273690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.273641    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:32:29.273690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.273654    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx" (OuterVolumeSpecName: "kube-api-access-xmwqx") pod "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" (UID: "06f8f82d-8b86-42dd-bbe7-0eaac2d4c303"). InnerVolumeSpecName "kube-api-access-xmwqx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:29.305662 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.305634    2573 generic.go:356] "Generic (PLEG): container finished" podID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerID="8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b" exitCode=0
Sep 11 13:32:29.305779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.305685    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"
Sep 11 13:32:29.305779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.305725    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerDied","Data":"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"}
Sep 11 13:32:29.305854 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.305786    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5" event={"ID":"06f8f82d-8b86-42dd-bbe7-0eaac2d4c303","Type":"ContainerDied","Data":"6443d705d48cf1e018fac4216a2f01b58e3348c5afc013f459dd758c91bd76a6"}
Sep 11 13:32:29.305854 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.305810    2573 scope.go:117] "RemoveContainer" containerID="8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"
Sep 11 13:32:29.314632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.314606    2573 scope.go:117] "RemoveContainer" containerID="56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"
Sep 11 13:32:29.322190 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.322175    2573 scope.go:117] "RemoveContainer" containerID="8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"
Sep 11 13:32:29.322434 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:29.322417    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b\": container with ID starting with 8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b not found: ID does not exist" containerID="8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"
Sep 11 13:32:29.322482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.322439    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b"} err="failed to get container status \"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b\": rpc error: code = NotFound desc = could not find container \"8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b\": container with ID starting with 8efe6e4bd4a9a1d458965dcc93696159fdc279335bf25dd56759e0238829129b not found: ID does not exist"
Sep 11 13:32:29.322482 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.322454    2573 scope.go:117] "RemoveContainer" containerID="56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"
Sep 11 13:32:29.322674 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:29.322661    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432\": container with ID starting with 56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432 not found: ID does not exist" containerID="56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"
Sep 11 13:32:29.322745 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.322678    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432"} err="failed to get container status \"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432\": rpc error: code = NotFound desc = could not find container \"56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432\": container with ID starting with 56521564184b24cbff901534f98b751866267881b13e10da1677c3dda0c4b432 not found: ID does not exist"
Sep 11 13:32:29.326970 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.326950    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:32:29.332276 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.332254    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w2-e0ebd7-openshift-default-77575b8b44-qrtp5"]
Sep 11 13:32:29.372759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372735    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.372759 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372759    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.372914 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372771    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.372914 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372785    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.372914 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372796    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.372914 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.372809    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmwqx\" (UniqueName: \"kubernetes.io/projected/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303-kube-api-access-xmwqx\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:29.972718 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:29.972679    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" path="/var/lib/kubelet/pods/06f8f82d-8b86-42dd-bbe7-0eaac2d4c303/volumes"
Sep 11 13:32:30.608940 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:30.608898    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:32:30.718083 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:30.718052    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:30.718233 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:30.718099    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:30.730168 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:30.730144    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:31.328116 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:31.328087    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:32:34.962008 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:34.961972    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:32:34.962547 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:34.962187    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" podUID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" containerName="maas-api" containerID="cri-o://6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb" gracePeriod=30
Sep 11 13:32:35.215836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.215777    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:32:35.323472 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.323442    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nlc62\" (UniqueName: \"kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62\") pod \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") "
Sep 11 13:32:35.323654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.323498    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") pod \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") "
Sep 11 13:32:35.323654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.323548    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca\") pod \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") "
Sep 11 13:32:35.323654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.323585    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs\") pod \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\" (UID: \"7671dc6e-de9f-4f02-883c-9ac69cd87a4f\") "
Sep 11 13:32:35.324011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.323978    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "7671dc6e-de9f-4f02-883c-9ac69cd87a4f" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:32:35.325676 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.325652    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "7671dc6e-de9f-4f02-883c-9ac69cd87a4f" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:32:35.325788 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.325675    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62" (OuterVolumeSpecName: "kube-api-access-nlc62") pod "7671dc6e-de9f-4f02-883c-9ac69cd87a4f" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f"). InnerVolumeSpecName "kube-api-access-nlc62". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:32:35.325788 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.325677    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7671dc6e-de9f-4f02-883c-9ac69cd87a4f" (UID: "7671dc6e-de9f-4f02-883c-9ac69cd87a4f"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:32:35.339368 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.339344    2573 generic.go:356] "Generic (PLEG): container finished" podID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" containerID="6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb" exitCode=0
Sep 11 13:32:35.339462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.339383    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" event={"ID":"7671dc6e-de9f-4f02-883c-9ac69cd87a4f","Type":"ContainerDied","Data":"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"}
Sep 11 13:32:35.339462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.339419    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6" event={"ID":"7671dc6e-de9f-4f02-883c-9ac69cd87a4f","Type":"ContainerDied","Data":"f45ee55376953dd02c7dcef51faa1a752f8e35c47e42d14edd52c990e2a3edc3"}
Sep 11 13:32:35.339462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.339440    2573 scope.go:117] "RemoveContainer" containerID="6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"
Sep 11 13:32:35.339462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.339384    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"
Sep 11 13:32:35.350313 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.349955    2573 scope.go:117] "RemoveContainer" containerID="6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"
Sep 11 13:32:35.351219 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:32:35.351169    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb\": container with ID starting with 6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb not found: ID does not exist" containerID="6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"
Sep 11 13:32:35.351332 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.351209    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb"} err="failed to get container status \"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb\": rpc error: code = NotFound desc = could not find container \"6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb\": container with ID starting with 6efba8169ca054170414ddd83fbb76d8558801436f8c445f659b64c5fe450ccb not found: ID does not exist"
Sep 11 13:32:35.370119 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.370095    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:32:35.374692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.374672    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-shared-b-w2-1eb525-856cf4b58c-qcqb6"]
Sep 11 13:32:35.424414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.424388    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.424414 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.424411    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.424553 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.424421    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nlc62\" (UniqueName: \"kubernetes.io/projected/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-kube-api-access-nlc62\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.424553 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.424429    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7671dc6e-de9f-4f02-883c-9ac69cd87a4f-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:32:35.972247 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:35.972213    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" path="/var/lib/kubelet/pods/7671dc6e-de9f-4f02-883c-9ac69cd87a4f/volumes"
Sep 11 13:32:41.002297 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:41.002261    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:32:56.917126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:32:56.917087    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:33:02.774266 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:02.774233    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:33:03.887723 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:03.887680    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:33:04.795690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.795659    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:33:04.796048 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796033    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:04.796098 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796051    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:04.796098 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796067    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" containerName="maas-api"
Sep 11 13:33:04.796098 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796072    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" containerName="maas-api"
Sep 11 13:33:04.796196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796146    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7671dc6e-de9f-4f02-883c-9ac69cd87a4f" containerName="maas-api"
Sep 11 13:33:04.796196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796159    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:04.796196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.796165    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:04.799487 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.799466    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.802090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.802067    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-a-48243b-openshift-default-dockercfg-m646f\""
Sep 11 13:33:04.820762 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.820740    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:33:04.877058 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877030    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877079    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877199 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877181    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6vtrm\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877236    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877330 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877293    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877369 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877348    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877383    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877450 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877414    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.877450 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.877436    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.978977 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.978950    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.978993    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979025    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979053    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979078    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979157    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979188    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979229    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6vtrm\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979436 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979265    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979448    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979672    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.979869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979730    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.980038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.979902    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.980099 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.980057    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.981409 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.981347    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.981644 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.981623    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.988126 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.988096    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6vtrm\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:04.988235 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:04.988172    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token\") pod \"e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") " pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:05.112308 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.112233    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:05.386960 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.386938    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:33:05.389125 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:33:05.389097    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb465727_6892_4c52_9dcd_c2a6b97f5ae7.slice/crio-1ba687e16c98528a7589204252147a4f81dbb7cf7d8645a05956755e34289a58 WatchSource:0}: Error finding container 1ba687e16c98528a7589204252147a4f81dbb7cf7d8645a05956755e34289a58: Status 404 returned error can't find the container with id 1ba687e16c98528a7589204252147a4f81dbb7cf7d8645a05956755e34289a58
Sep 11 13:33:05.391452 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.391417    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:05.391553 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.391485    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:05.391553 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.391540    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:05.503181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.503149    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerStarted","Data":"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"}
Sep 11 13:33:05.503360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.503199    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerStarted","Data":"1ba687e16c98528a7589204252147a4f81dbb7cf7d8645a05956755e34289a58"}
Sep 11 13:33:05.524901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:05.524644    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podStartSLOduration=1.5245796550000001 podStartE2EDuration="1.524579655s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:05.524306208 +0000 UTC m=+1700.044343554" watchObservedRunningTime="2026-09-11 13:33:05.524579655 +0000 UTC m=+1700.044616978"
Sep 11 13:33:06.113221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:06.113191    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:06.118511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:06.118483    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:06.508019 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:06.507991    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:06.508961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:06.508943    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:07.717799 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:07.717765    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:33:10.279754 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.279668    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:33:10.280095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.280054    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:10.280095 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.280067    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="06f8f82d-8b86-42dd-bbe7-0eaac2d4c303" containerName="istio-proxy"
Sep 11 13:33:10.283360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.283344    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.286117 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.286094    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-e2e-ipp-a-48243b-dockercfg-9lzlt\""
Sep 11 13:33:10.286274 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.286179    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-a-48243b\""
Sep 11 13:33:10.299806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.299784    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:33:10.429015 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.428978    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.429178 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.429080    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8dfcr\" (UniqueName: \"kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.530116 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.530037    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.530250 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.530193    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8dfcr\" (UniqueName: \"kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.530691 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.530672    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.539009 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.538989    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8dfcr\" (UniqueName: \"kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr\") pod \"payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") " pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.594964 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.594932    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:10.865876 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:10.865798    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:33:10.869044 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:33:10.869017    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b7b0c53_9fce_49a5_be5a_69c35c2bb95b.slice/crio-1651ae55f90eb7a328b0832275685a5212c739895b5c8069f2f1544c8ac2fccb WatchSource:0}: Error finding container 1651ae55f90eb7a328b0832275685a5212c739895b5c8069f2f1544c8ac2fccb: Status 404 returned error can't find the container with id 1651ae55f90eb7a328b0832275685a5212c739895b5c8069f2f1544c8ac2fccb
Sep 11 13:33:11.538305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:11.538263    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" event={"ID":"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b","Type":"ContainerStarted","Data":"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"}
Sep 11 13:33:11.538752 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:11.538328    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" event={"ID":"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b","Type":"ContainerStarted","Data":"1651ae55f90eb7a328b0832275685a5212c739895b5c8069f2f1544c8ac2fccb"}
Sep 11 13:33:11.538752 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:11.538401    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:11.557953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:11.557909    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" podStartSLOduration=1.557895876 podStartE2EDuration="1.557895876s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:11.555193757 +0000 UTC m=+1706.075231102" watchObservedRunningTime="2026-09-11 13:33:11.557895876 +0000 UTC m=+1706.077933255"
Sep 11 13:33:17.792999 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:17.792961    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" podUID="32264fbb-bbff-4754-963e-e0eb2db8e37b" containerName="main" containerID="cri-o://04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3" gracePeriod=45
Sep 11 13:33:18.083005 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.082979    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:33:18.102226 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102206    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102238    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102292    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102314    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-l5z7f\" (UniqueName: \"kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102339    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102414    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache\") pod \"32264fbb-bbff-4754-963e-e0eb2db8e37b\" (UID: \"32264fbb-bbff-4754-963e-e0eb2db8e37b\") "
Sep 11 13:33:18.102584 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102487    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:18.102661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102633    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home" (OuterVolumeSpecName: "home") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:18.102780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102754    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache" (OuterVolumeSpecName: "model-cache") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:18.102848 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102776    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.102848 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.102798    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.104319 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.104297    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:33:18.104403 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.104386    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm" (OuterVolumeSpecName: "dshm") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:18.104630 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.104612    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f" (OuterVolumeSpecName: "kube-api-access-l5z7f") pod "32264fbb-bbff-4754-963e-e0eb2db8e37b" (UID: "32264fbb-bbff-4754-963e-e0eb2db8e37b"). InnerVolumeSpecName "kube-api-access-l5z7f". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:33:18.204193 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.204163    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l5z7f\" (UniqueName: \"kubernetes.io/projected/32264fbb-bbff-4754-963e-e0eb2db8e37b-kube-api-access-l5z7f\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.204193 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.204190    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.204383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.204202    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/32264fbb-bbff-4754-963e-e0eb2db8e37b-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.204383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.204217    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/32264fbb-bbff-4754-963e-e0eb2db8e37b-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:18.418334 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.418246    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:33:18.579194 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.579157    2573 generic.go:356] "Generic (PLEG): container finished" podID="32264fbb-bbff-4754-963e-e0eb2db8e37b" containerID="04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3" exitCode=0
Sep 11 13:33:18.579194 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.579196    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" event={"ID":"32264fbb-bbff-4754-963e-e0eb2db8e37b","Type":"ContainerDied","Data":"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"}
Sep 11 13:33:18.579427 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.579204    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"
Sep 11 13:33:18.579427 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.579223    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86" event={"ID":"32264fbb-bbff-4754-963e-e0eb2db8e37b","Type":"ContainerDied","Data":"ad927168553d90810ecc68a0400686362cfc1a18bcdad69d688b37eb72a28c99"}
Sep 11 13:33:18.579427 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.579241    2573 scope.go:117] "RemoveContainer" containerID="04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"
Sep 11 13:33:18.588586 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.588561    2573 scope.go:117] "RemoveContainer" containerID="04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"
Sep 11 13:33:18.588862 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:33:18.588843    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3\": container with ID starting with 04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3 not found: ID does not exist" containerID="04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"
Sep 11 13:33:18.588920 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.588869    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3"} err="failed to get container status \"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3\": rpc error: code = NotFound desc = could not find container \"04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3\": container with ID starting with 04b2f3d927555978979a4b8cb0e0ad82a005ba0d927ae552fc83c5a5999a65d3 not found: ID does not exist"
Sep 11 13:33:18.602280 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.602260    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:33:18.607124 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.607104    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-w3-6ffb6a/rate-test-model-6ffb6a-kserve-648bfc89f9-jgw86"]
Sep 11 13:33:18.906137 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:18.906087    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" podUID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" containerName="main" containerID="cri-o://09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac" gracePeriod=45
Sep 11 13:33:19.197599 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.197573    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:33:19.211953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.211932    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.211986    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212018    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212046    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212080 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212063    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212105    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-whvtd\" (UniqueName: \"kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd\") pod \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\" (UID: \"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9\") "
Sep 11 13:33:19.212309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212165    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:19.212309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212245    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home" (OuterVolumeSpecName: "home") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:19.212309 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212258    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache" (OuterVolumeSpecName: "model-cache") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:19.212501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212414    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.212501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212433    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.212501 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.212448    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.214177 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.214154    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd" (OuterVolumeSpecName: "kube-api-access-whvtd") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "kube-api-access-whvtd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:33:19.214296 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.214208    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:33:19.214296 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.214243    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm" (OuterVolumeSpecName: "dshm") pod "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" (UID: "c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:33:19.313530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.313498    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.313530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.313524    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.313530 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.313533    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-whvtd\" (UniqueName: \"kubernetes.io/projected/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9-kube-api-access-whvtd\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:33:19.586571 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.586538    2573 generic.go:356] "Generic (PLEG): container finished" podID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" containerID="09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac" exitCode=0
Sep 11 13:33:19.586766 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.586578    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"
Sep 11 13:33:19.586766 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.586629    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" event={"ID":"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9","Type":"ContainerDied","Data":"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"}
Sep 11 13:33:19.586766 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.586671    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc" event={"ID":"c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9","Type":"ContainerDied","Data":"ce439442b91b46d47c52a3ee9a56f522b47a5d676acc59bbcd36fe468bec8469"}
Sep 11 13:33:19.586766 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.586691    2573 scope.go:117] "RemoveContainer" containerID="09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"
Sep 11 13:33:19.595572 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.595557    2573 scope.go:117] "RemoveContainer" containerID="09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"
Sep 11 13:33:19.595849 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:33:19.595831    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac\": container with ID starting with 09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac not found: ID does not exist" containerID="09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"
Sep 11 13:33:19.595901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.595858    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac"} err="failed to get container status \"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac\": rpc error: code = NotFound desc = could not find container \"09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac\": container with ID starting with 09cab3c2dd5d5244eaa98eb9832cd60f7f6e7d8ec3a23b32651869020e9789ac not found: ID does not exist"
Sep 11 13:33:19.611039 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.611014    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:33:19.614794 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.614776    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-w3-12f386/rate-test-model-12f386-kserve-58f989d4b7-rhcdc"]
Sep 11 13:33:19.972161 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.972065    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="32264fbb-bbff-4754-963e-e0eb2db8e37b" path="/var/lib/kubelet/pods/32264fbb-bbff-4754-963e-e0eb2db8e37b/volumes"
Sep 11 13:33:19.972670 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:19.972650    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" path="/var/lib/kubelet/pods/c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9/volumes"
Sep 11 13:33:22.545898 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:22.545861    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:33:22.550843 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:22.550814    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body=
Sep 11 13:33:22.550958 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:22.550869    2573 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)"
Sep 11 13:33:24.615027 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.614996    2573 generic.go:356] "Generic (PLEG): container finished" podID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerID="01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb" exitCode=0
Sep 11 13:33:24.615416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.615037    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerDied","Data":"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"}
Sep 11 13:33:24.615416 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.615403    2573 scope.go:117] "RemoveContainer" containerID="01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"
Sep 11 13:33:24.616525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.616496    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:24.616603 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.616588    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:24.616641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:24.616629    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:25.621815 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:25.621780    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerStarted","Data":"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"}
Sep 11 13:33:26.112902 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:26.112868    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:26.113104 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:26.113081    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 11 13:33:26.113165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:26.113135    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 11 13:33:27.112977 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:27.112939    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 11 13:33:27.113346 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:27.113009    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 11 13:33:28.113072 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:28.113037    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 11 13:33:28.113441 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:28.113093    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 11 13:33:29.113172 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:29.113136    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 11 13:33:29.113554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:29.113187    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 11 13:33:30.113366 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:30.113332    2573 patch_prober.go:28] interesting pod/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused" start-of-body=
Sep 11 13:33:30.113795 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:30.113396    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.81:15021/healthz/ready\": dial tcp 10.134.0.81:15021: connect: connection refused"
Sep 11 13:33:31.122275 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:31.122245    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:31.122772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:31.122680    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:31.123431 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:31.123408    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:33:33.309296 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309261    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:33:33.309824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309801    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="32264fbb-bbff-4754-963e-e0eb2db8e37b" containerName="main"
Sep 11 13:33:33.309824 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309823    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="32264fbb-bbff-4754-963e-e0eb2db8e37b" containerName="main"
Sep 11 13:33:33.309942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309842    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" containerName="main"
Sep 11 13:33:33.309942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309851    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" containerName="main"
Sep 11 13:33:33.309942 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309932    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="32264fbb-bbff-4754-963e-e0eb2db8e37b" containerName="main"
Sep 11 13:33:33.310061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.309945    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7d8fb06-d4c7-4aee-9a4b-faa64816b2c9" containerName="main"
Sep 11 13:33:33.312235 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.312213    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.315022 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.315001    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-ipp-b-47b957-openshift-default-dockercfg-xbwxm\""
Sep 11 13:33:33.326418 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.326397    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:33:33.446061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446022    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446076    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bcwz7\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446140    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446167    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446221 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446200    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446241    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446263    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446287    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.446357 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.446321    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.546863 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.546830    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.546863 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.546864    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bcwz7\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.546906    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.546924    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547108    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547132    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547155    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547193    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547417    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547502 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547483    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547556    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.547661 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547570    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.548004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.547981    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.549286 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.549263    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.549597 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.549575    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.554571 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.554543    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.554965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.554945    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bcwz7\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7\") pod \"e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") " pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.622465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.622396    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:33.903001 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.902974    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:33:33.905749 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:33:33.905717    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd4c1284e_6fcf_49f3_9f2b_beca24fddb74.slice/crio-992b44ec51a3e59e44f176faa6cafb3f3633ddde749523b48db1b1ddff7bf269 WatchSource:0}: Error finding container 992b44ec51a3e59e44f176faa6cafb3f3633ddde749523b48db1b1ddff7bf269: Status 404 returned error can't find the container with id 992b44ec51a3e59e44f176faa6cafb3f3633ddde749523b48db1b1ddff7bf269
Sep 11 13:33:33.908088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.908050    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:33.908181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.908123    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:33.908181 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:33.908154    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:34.675950 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:34.675916    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerStarted","Data":"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"}
Sep 11 13:33:34.676318 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:34.675978    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerStarted","Data":"992b44ec51a3e59e44f176faa6cafb3f3633ddde749523b48db1b1ddff7bf269"}
Sep 11 13:33:34.700293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:34.700205    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podStartSLOduration=1.7001909880000001 podStartE2EDuration="1.700190988s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:33:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:33:34.697384213 +0000 UTC m=+1729.217421558" watchObservedRunningTime="2026-09-11 13:33:34.700190988 +0000 UTC m=+1729.220228333"
Sep 11 13:33:35.622981 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:35.622949    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:35.627947 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:35.627920    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:35.681814 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:35.681786    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:35.682700 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:35.682678    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:44.296011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:44.295969    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:33:54.794167 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.794134    2573 generic.go:356] "Generic (PLEG): container finished" podID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerID="cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee" exitCode=0
Sep 11 13:33:54.794541 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.794205    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerDied","Data":"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"}
Sep 11 13:33:54.794606 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.794592    2573 scope.go:117] "RemoveContainer" containerID="cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"
Sep 11 13:33:54.795655 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.795628    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:54.795733 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.795699    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:54.795777 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:54.795760    2573 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236208Ki","pods":"250"}
Sep 11 13:33:55.800665 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:55.800627    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerStarted","Data":"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"}
Sep 11 13:33:56.623383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:56.623346    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:33:56.623565 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:56.623525    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:33:56.623614 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:56.623572    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 11 13:33:57.623057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:57.623019    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:33:57.623422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:57.623076    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 11 13:33:58.623050 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:58.623016    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:33:58.623219 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:58.623078    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 11 13:33:59.623161 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:59.623122    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:33:59.623346 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:33:59.623192    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 11 13:34:00.623066 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:00.623030    2573 patch_prober.go:28] interesting pod/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused" start-of-body=
Sep 11 13:34:00.623239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:00.623080    2573 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.134.0.83:15021/healthz/ready\": dial tcp 10.134.0.83:15021: connect: connection refused"
Sep 11 13:34:01.626721 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:01.626679    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:34:01.627100 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:01.626996    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:34:01.627578 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:01.627559    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:34:06.550283 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:06.550257    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:34:32.906413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.906370    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:34:32.908931 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.908909    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:32.911196 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.911177    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-48243b\"/\"default-dockercfg-jnrcj\""
Sep 11 13:34:32.911779 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.911760    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-48243b\"/\"openshift-service-ca.crt\""
Sep 11 13:34:32.911881 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.911779    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-ipp-a-48243b\"/\"kube-root-ca.crt\""
Sep 11 13:34:32.913546 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.913528    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-ipp-a-48243b\"/\"ipp-route-48243b-kserve-self-signed-certs\""
Sep 11 13:34:32.918305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:32.918283    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:34:33.071444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071415    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.071611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071468    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.071611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071502    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.071611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071533    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.071611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071561    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbpft\" (UniqueName: \"kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.071820 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.071666    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.172927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.172859    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.172927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.172891    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.172927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.172911    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.172933    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbpft\" (UniqueName: \"kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.172978    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.173048    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173289 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.173268    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173343 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.173325    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.173475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.173393    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.175782 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.175759    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.177326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.177299    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.183782 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.183759    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbpft\" (UniqueName: \"kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft\") pod \"ipp-route-48243b-kserve-555d88dccf-l5d2n\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") " pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.218871 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.218841    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:33.488851 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:33.488826    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:34:33.490873 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:34:33.490836    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod75a224d4_595f_4ae2_8a02_35d6a9d67adf.slice/crio-75c3fa8ef6c15e0aca975dab3c0ee7154454d23a60f05872aa7a72e0f0594faa WatchSource:0}: Error finding container 75c3fa8ef6c15e0aca975dab3c0ee7154454d23a60f05872aa7a72e0f0594faa: Status 404 returned error can't find the container with id 75c3fa8ef6c15e0aca975dab3c0ee7154454d23a60f05872aa7a72e0f0594faa
Sep 11 13:34:34.009680 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:34.009639    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" event={"ID":"75a224d4-595f-4ae2-8a02-35d6a9d67adf","Type":"ContainerStarted","Data":"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"}
Sep 11 13:34:34.010063 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:34.009689    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" event={"ID":"75a224d4-595f-4ae2-8a02-35d6a9d67adf","Type":"ContainerStarted","Data":"75c3fa8ef6c15e0aca975dab3c0ee7154454d23a60f05872aa7a72e0f0594faa"}
Sep 11 13:34:34.029105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:34.029057    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" podStartSLOduration=2.029043486 podStartE2EDuration="2.029043486s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:34:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:34:34.027320146 +0000 UTC m=+1788.547357485" watchObservedRunningTime="2026-09-11 13:34:34.029043486 +0000 UTC m=+1788.549080832"
Sep 11 13:34:36.897463 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:36.897426    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:34:43.219165 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:43.219078    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:43.219563 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:43.219326    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:43.231544 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:43.231519    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:34:44.071455 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:34:44.071415    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:35:06.805683 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:06.805651    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:35:13.049483 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:13.049453    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz_8b7b0c53-9fce-49a5-be5a-69c35c2bb95b/payload-processing/0.log"
Sep 11 13:35:13.173894 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:13.173862    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:35:24.023686 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.023454    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:35:24.026974 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.026949    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.029251 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.029217    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-ipp-cleanup-d5e184-serving-cert\""
Sep 11 13:35:24.040353 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.040324    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:35:24.116133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.116103    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.116298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.116155    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.116298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.116220    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhv8q\" (UniqueName: \"kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.116298 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.116286    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.217507 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.217470    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.217688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.217536    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.217688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.217607    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhv8q\" (UniqueName: \"kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.217688 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:35:24.217633    2573 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-serving-cert: secret "maas-api-e2e-ipp-cleanup-d5e184-serving-cert" not found
Sep 11 13:35:24.217688 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.217666    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.217946 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:35:24.217698    2573 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls podName:2cb598fe-1de5-4f74-8bf0-eef0eb463be4 nodeName:}" failed. No retries permitted until 2026-09-11 13:35:24.717677978 +0000 UTC m=+1839.237715314 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls") pod "maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" (UID: "2cb598fe-1de5-4f74-8bf0-eef0eb463be4") : secret "maas-api-e2e-ipp-cleanup-d5e184-serving-cert" not found
Sep 11 13:35:24.218325 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.218308    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.220070 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.220053    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.238328 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.238303    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhv8q\" (UniqueName: \"kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.722636 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.722602    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.724995 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.724977    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") pod \"maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") " pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:24.745175 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.745143    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:35:24.749144 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.749123    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.751963 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.751937    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins-e2e-ipp-cleanup-d5e184\""
Sep 11 13:35:24.758168 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.757948    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:35:24.823800 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.823767    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7467d\" (UniqueName: \"kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.823937 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.823811    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.924659 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.924621    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7467d\" (UniqueName: \"kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.924840 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.924675    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.925381 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.925358    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.933641 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.933613    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7467d\" (UniqueName: \"kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d\") pod \"payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") " pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:24.938888 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:24.938863    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:25.062173 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:25.062144    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:25.214404 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:25.214378    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:35:25.215330 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:35:25.215298    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2cb598fe_1de5_4f74_8bf0_eef0eb463be4.slice/crio-7dbb8ddf8c7df9cba23e6803083f5ad7aa5bc1cb8c873412b01e9a2eea20131b WatchSource:0}: Error finding container 7dbb8ddf8c7df9cba23e6803083f5ad7aa5bc1cb8c873412b01e9a2eea20131b: Status 404 returned error can't find the container with id 7dbb8ddf8c7df9cba23e6803083f5ad7aa5bc1cb8c873412b01e9a2eea20131b
Sep 11 13:35:25.216916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:25.216898    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:35:25.264591 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:25.264568    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" event={"ID":"2cb598fe-1de5-4f74-8bf0-eef0eb463be4","Type":"ContainerStarted","Data":"7dbb8ddf8c7df9cba23e6803083f5ad7aa5bc1cb8c873412b01e9a2eea20131b"}
Sep 11 13:35:25.335308 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:35:25.335285    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod588675c9_4c53_4297_902f_8743e0a2a17b.slice/crio-c9763017f8071cd8f9a3ca6b4da92725cf0b8f75cc4f2dc9a3ac89114e826d3d WatchSource:0}: Error finding container c9763017f8071cd8f9a3ca6b4da92725cf0b8f75cc4f2dc9a3ac89114e826d3d: Status 404 returned error can't find the container with id c9763017f8071cd8f9a3ca6b4da92725cf0b8f75cc4f2dc9a3ac89114e826d3d
Sep 11 13:35:25.335873 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:25.335851    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:35:26.271986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.271952    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" event={"ID":"588675c9-4c53-4297-902f-8743e0a2a17b","Type":"ContainerStarted","Data":"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"}
Sep 11 13:35:26.272392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.272031    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:35:26.272392 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.272056    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" event={"ID":"588675c9-4c53-4297-902f-8743e0a2a17b","Type":"ContainerStarted","Data":"c9763017f8071cd8f9a3ca6b4da92725cf0b8f75cc4f2dc9a3ac89114e826d3d"}
Sep 11 13:35:26.273586 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.273554    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" event={"ID":"2cb598fe-1de5-4f74-8bf0-eef0eb463be4","Type":"ContainerStarted","Data":"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"}
Sep 11 13:35:26.273752 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.273606    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:26.309174 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.309132    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" podStartSLOduration=2.309119273 podStartE2EDuration="2.309119273s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:24 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:26.305049057 +0000 UTC m=+1840.825086402" watchObservedRunningTime="2026-09-11 13:35:26.309119273 +0000 UTC m=+1840.829156686"
Sep 11 13:35:26.332092 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:26.332056    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" podStartSLOduration=2.950464046 podStartE2EDuration="3.332043566s" totalImagesPullingTime="381.57952ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:35:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:35:26.329419302 +0000 UTC m=+1840.849456647" watchObservedRunningTime="2026-09-11 13:35:26.332043566 +0000 UTC m=+1840.852080911"
Sep 11 13:35:29.010722 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:29.010662    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:35:32.283491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:32.283462    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:35:37.280821 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:35:37.280786    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:36:04.039439 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.039406    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:36:04.039936 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.039622    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" podUID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" containerName="maas-api" containerID="cri-o://8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a" gracePeriod=30
Sep 11 13:36:04.185369 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.185330    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:36:04.185690 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.185568    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" podUID="588675c9-4c53-4297-902f-8743e0a2a17b" containerName="payload-processing" containerID="cri-o://c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d" gracePeriod=30
Sep 11 13:36:04.310886 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.310861    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:36:04.364187 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.364156    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hhv8q\" (UniqueName: \"kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q\") pod \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") "
Sep 11 13:36:04.364320 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.364243    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") pod \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") "
Sep 11 13:36:04.364371 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.364361    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs\") pod \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") "
Sep 11 13:36:04.364413 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.364389    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca\") pod \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\" (UID: \"2cb598fe-1de5-4f74-8bf0-eef0eb463be4\") "
Sep 11 13:36:04.364878 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.364851    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "2cb598fe-1de5-4f74-8bf0-eef0eb463be4" (UID: "2cb598fe-1de5-4f74-8bf0-eef0eb463be4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:04.367006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.366978    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "2cb598fe-1de5-4f74-8bf0-eef0eb463be4" (UID: "2cb598fe-1de5-4f74-8bf0-eef0eb463be4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:36:04.367006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.366996    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q" (OuterVolumeSpecName: "kube-api-access-hhv8q") pod "2cb598fe-1de5-4f74-8bf0-eef0eb463be4" (UID: "2cb598fe-1de5-4f74-8bf0-eef0eb463be4"). InnerVolumeSpecName "kube-api-access-hhv8q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:04.367152 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.367091    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "2cb598fe-1de5-4f74-8bf0-eef0eb463be4" (UID: "2cb598fe-1de5-4f74-8bf0-eef0eb463be4"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:36:04.436737 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.436697    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:36:04.465522 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.465493    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hhv8q\" (UniqueName: \"kubernetes.io/projected/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-kube-api-access-hhv8q\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.465522 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.465521    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.465732 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.465535    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.465732 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.465548    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2cb598fe-1de5-4f74-8bf0-eef0eb463be4-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.478658 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.478625    2573 generic.go:356] "Generic (PLEG): container finished" podID="588675c9-4c53-4297-902f-8743e0a2a17b" containerID="c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d" exitCode=0
Sep 11 13:36:04.478813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.478661    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"
Sep 11 13:36:04.478813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.478727    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" event={"ID":"588675c9-4c53-4297-902f-8743e0a2a17b","Type":"ContainerDied","Data":"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"}
Sep 11 13:36:04.478813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.478784    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz" event={"ID":"588675c9-4c53-4297-902f-8743e0a2a17b","Type":"ContainerDied","Data":"c9763017f8071cd8f9a3ca6b4da92725cf0b8f75cc4f2dc9a3ac89114e826d3d"}
Sep 11 13:36:04.478813 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.478811    2573 scope.go:117] "RemoveContainer" containerID="c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"
Sep 11 13:36:04.480573 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.480552    2573 generic.go:356] "Generic (PLEG): container finished" podID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" containerID="8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a" exitCode=0
Sep 11 13:36:04.480675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.480584    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" event={"ID":"2cb598fe-1de5-4f74-8bf0-eef0eb463be4","Type":"ContainerDied","Data":"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"}
Sep 11 13:36:04.480675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.480608    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"
Sep 11 13:36:04.480675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.480631    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4" event={"ID":"2cb598fe-1de5-4f74-8bf0-eef0eb463be4","Type":"ContainerDied","Data":"7dbb8ddf8c7df9cba23e6803083f5ad7aa5bc1cb8c873412b01e9a2eea20131b"}
Sep 11 13:36:04.494016 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.493825    2573 scope.go:117] "RemoveContainer" containerID="c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"
Sep 11 13:36:04.494152 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:36:04.494132    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d\": container with ID starting with c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d not found: ID does not exist" containerID="c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"
Sep 11 13:36:04.494220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.494160    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d"} err="failed to get container status \"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d\": rpc error: code = NotFound desc = could not find container \"c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d\": container with ID starting with c492b8cac6d6006701c3e1b349d73c0c0e24e8096f25b4e20a16d8faf9d8957d not found: ID does not exist"
Sep 11 13:36:04.494220 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.494175    2573 scope.go:117] "RemoveContainer" containerID="8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"
Sep 11 13:36:04.502408 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.502389    2573 scope.go:117] "RemoveContainer" containerID="8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"
Sep 11 13:36:04.502649 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:36:04.502630    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a\": container with ID starting with 8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a not found: ID does not exist" containerID="8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"
Sep 11 13:36:04.502693 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.502654    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a"} err="failed to get container status \"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a\": rpc error: code = NotFound desc = could not find container \"8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a\": container with ID starting with 8c1b6f6212edf86abc524c7fa9d5281f7dcbc03b452099a183e57a8092add16a not found: ID does not exist"
Sep 11 13:36:04.521569 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.521543    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:36:04.525105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.525086    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-ipp-cleanup-d5e184-65f4987f88-fghg4"]
Sep 11 13:36:04.566173 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.566094    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume\") pod \"588675c9-4c53-4297-902f-8743e0a2a17b\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") "
Sep 11 13:36:04.566173 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.566162    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-7467d\" (UniqueName: \"kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d\") pod \"588675c9-4c53-4297-902f-8743e0a2a17b\" (UID: \"588675c9-4c53-4297-902f-8743e0a2a17b\") "
Sep 11 13:36:04.566533 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.566506    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "588675c9-4c53-4297-902f-8743e0a2a17b" (UID: "588675c9-4c53-4297-902f-8743e0a2a17b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:04.568184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.568162    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d" (OuterVolumeSpecName: "kube-api-access-7467d") pod "588675c9-4c53-4297-902f-8743e0a2a17b" (UID: "588675c9-4c53-4297-902f-8743e0a2a17b"). InnerVolumeSpecName "kube-api-access-7467d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:04.667326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.667295    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/588675c9-4c53-4297-902f-8743e0a2a17b-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.667326 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.667322    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7467d\" (UniqueName: \"kubernetes.io/projected/588675c9-4c53-4297-902f-8743e0a2a17b-kube-api-access-7467d\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:04.818341 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.818280    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:36:04.855516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:04.855493    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-cleanup-d5e184-5977d4f6c7-b27gz"]
Sep 11 13:36:05.977421 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:05.977383    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" path="/var/lib/kubelet/pods/2cb598fe-1de5-4f74-8bf0-eef0eb463be4/volumes"
Sep 11 13:36:05.977803 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:05.977755    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="588675c9-4c53-4297-902f-8743e0a2a17b" path="/var/lib/kubelet/pods/588675c9-4c53-4297-902f-8743e0a2a17b/volumes"
Sep 11 13:36:08.515120 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:08.515020    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:36:28.496760 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:28.496729    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:36:30.355838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.355805    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:36:30.356342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.356025    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" podUID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" containerName="payload-processing" containerID="cri-o://2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e" gracePeriod=30
Sep 11 13:36:30.612965 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.612911    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:36:30.622992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.622967    2573 generic.go:356] "Generic (PLEG): container finished" podID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" containerID="2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e" exitCode=0
Sep 11 13:36:30.623113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.623005    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"
Sep 11 13:36:30.623113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.623024    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" event={"ID":"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b","Type":"ContainerDied","Data":"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"}
Sep 11 13:36:30.623113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.623067    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz" event={"ID":"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b","Type":"ContainerDied","Data":"1651ae55f90eb7a328b0832275685a5212c739895b5c8069f2f1544c8ac2fccb"}
Sep 11 13:36:30.623113 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.623094    2573 scope.go:117] "RemoveContainer" containerID="2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"
Sep 11 13:36:30.631654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.631632    2573 scope.go:117] "RemoveContainer" containerID="2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"
Sep 11 13:36:30.632014 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:36:30.631985    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e\": container with ID starting with 2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e not found: ID does not exist" containerID="2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"
Sep 11 13:36:30.632102 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.632026    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e"} err="failed to get container status \"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e\": rpc error: code = NotFound desc = could not find container \"2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e\": container with ID starting with 2d6fc97dbe4cc9c528c18a170e5931d04059efb0916e27c30ec54381b1b8b63e not found: ID does not exist"
Sep 11 13:36:30.803903 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.803872    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume\") pod \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") "
Sep 11 13:36:30.804077 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.803922    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8dfcr\" (UniqueName: \"kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr\") pod \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\" (UID: \"8b7b0c53-9fce-49a5-be5a-69c35c2bb95b\") "
Sep 11 13:36:30.804254 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.804232    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume" (OuterVolumeSpecName: "plugins-config-volume") pod "8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" (UID: "8b7b0c53-9fce-49a5-be5a-69c35c2bb95b"). InnerVolumeSpecName "plugins-config-volume". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:36:30.806057 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.806033    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr" (OuterVolumeSpecName: "kube-api-access-8dfcr") pod "8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" (UID: "8b7b0c53-9fce-49a5-be5a-69c35c2bb95b"). InnerVolumeSpecName "kube-api-access-8dfcr". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:30.904989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.904921    2573 reconciler_common.go:299] "Volume detached for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-plugins-config-volume\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.904989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.904945    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8dfcr\" (UniqueName: \"kubernetes.io/projected/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b-kube-api-access-8dfcr\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:30.945929 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.945901    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:36:30.948279 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:30.948255    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/payload-processing-e2e-ipp-a-48243b-8d55b987-gvwmz"]
Sep 11 13:36:31.972305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:31.972269    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" path="/var/lib/kubelet/pods/8b7b0c53-9fce-49a5-be5a-69c35c2bb95b/volumes"
Sep 11 13:36:35.896740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:35.896694    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:36:40.833961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:40.833928    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:36:46.231719 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:46.231680    2573 scope.go:117] "RemoveContainer" containerID="8f5b6e1a09382b2aafd10c1b34a9044a3b1bda03b534263c94702e5ab7569859"
Sep 11 13:36:50.915691 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:50.915635    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" podUID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" containerName="main" containerID="cri-o://bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054" gracePeriod=45
Sep 11 13:36:51.162049 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.162025    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:36:51.273105 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273076    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273123    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273206    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273271 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273235    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273446 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273280    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273446 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273310    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:51.273446 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273323    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lbpft\" (UniqueName: \"kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft\") pod \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\" (UID: \"75a224d4-595f-4ae2-8a02-35d6a9d67adf\") "
Sep 11 13:36:51.273446 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273390    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home" (OuterVolumeSpecName: "home") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:51.273649 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273511    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache" (OuterVolumeSpecName: "model-cache") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:51.273724 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273692    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.273772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273738    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.273772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.273751    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.275492 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.275469    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm" (OuterVolumeSpecName: "dshm") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:36:51.275492 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.275479    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:36:51.275605 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.275516    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft" (OuterVolumeSpecName: "kube-api-access-lbpft") pod "75a224d4-595f-4ae2-8a02-35d6a9d67adf" (UID: "75a224d4-595f-4ae2-8a02-35d6a9d67adf"). InnerVolumeSpecName "kube-api-access-lbpft". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:36:51.374951 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.374928    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/75a224d4-595f-4ae2-8a02-35d6a9d67adf-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.374951 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.374949    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/75a224d4-595f-4ae2-8a02-35d6a9d67adf-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.375090 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.374961    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lbpft\" (UniqueName: \"kubernetes.io/projected/75a224d4-595f-4ae2-8a02-35d6a9d67adf-kube-api-access-lbpft\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:36:51.743284 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.743248    2573 generic.go:356] "Generic (PLEG): container finished" podID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" containerID="bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054" exitCode=0
Sep 11 13:36:51.743442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.743294    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"
Sep 11 13:36:51.743442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.743335    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" event={"ID":"75a224d4-595f-4ae2-8a02-35d6a9d67adf","Type":"ContainerDied","Data":"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"}
Sep 11 13:36:51.743442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.743384    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n" event={"ID":"75a224d4-595f-4ae2-8a02-35d6a9d67adf","Type":"ContainerDied","Data":"75c3fa8ef6c15e0aca975dab3c0ee7154454d23a60f05872aa7a72e0f0594faa"}
Sep 11 13:36:51.743442 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.743406    2573 scope.go:117] "RemoveContainer" containerID="bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"
Sep 11 13:36:51.752261 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.752243    2573 scope.go:117] "RemoveContainer" containerID="bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"
Sep 11 13:36:51.752508 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:36:51.752485    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054\": container with ID starting with bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054 not found: ID does not exist" containerID="bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"
Sep 11 13:36:51.752567 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.752514    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054"} err="failed to get container status \"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054\": rpc error: code = NotFound desc = could not find container \"bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054\": container with ID starting with bf52ed2f93638d28076e517fe2504d7c82af46fcf3febd256dd4557a9247f054 not found: ID does not exist"
Sep 11 13:36:51.765457 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.765435    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:36:51.776277 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.776258    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-ipp-a-48243b/ipp-route-48243b-kserve-555d88dccf-l5d2n"]
Sep 11 13:36:51.972204 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:36:51.972176    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" path="/var/lib/kubelet/pods/75a224d4-595f-4ae2-8a02-35d6a9d67adf/volumes"
Sep 11 13:37:19.990354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:19.990321    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:37:19.990953 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:19.990533    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy" containerID="cri-o://fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84" gracePeriod=30
Sep 11 13:37:25.237692 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.237669    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:37:25.359516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359448    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6vtrm\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359516 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359497    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359537    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359568    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359595    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359623    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359662    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359692    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.359852 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359751    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs\") pod \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\" (UID: \"bb465727-6892-4c52-9dcd-c2a6b97f5ae7\") "
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359852    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.359925    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360057    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data" (OuterVolumeSpecName: "istio-data") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360099    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360147    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360167    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360175    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:25.360205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.360182    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.361908 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.361879    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:37:25.362143 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.362069    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:25.362210 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.362191    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm" (OuterVolumeSpecName: "kube-api-access-6vtrm") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "kube-api-access-6vtrm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:25.362250 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.362210    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token" (OuterVolumeSpecName: "istio-token") pod "bb465727-6892-4c52-9dcd-c2a6b97f5ae7" (UID: "bb465727-6892-4c52-9dcd-c2a6b97f5ae7"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:25.460992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.460970    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6vtrm\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-kube-api-access-6vtrm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.460992 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.460990    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.461110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.460999    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.461110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.461007    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.461110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.461014    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.461110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.461021    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/bb465727-6892-4c52-9dcd-c2a6b97f5ae7-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:25.923465 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.923439    2573 generic.go:356] "Generic (PLEG): container finished" podID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerID="fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84" exitCode=0
Sep 11 13:37:25.923634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.923494    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"
Sep 11 13:37:25.923634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.923524    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerDied","Data":"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"}
Sep 11 13:37:25.923634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.923587    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8" event={"ID":"bb465727-6892-4c52-9dcd-c2a6b97f5ae7","Type":"ContainerDied","Data":"1ba687e16c98528a7589204252147a4f81dbb7cf7d8645a05956755e34289a58"}
Sep 11 13:37:25.923634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.923614    2573 scope.go:117] "RemoveContainer" containerID="fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"
Sep 11 13:37:25.933302 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.933283    2573 scope.go:117] "RemoveContainer" containerID="01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"
Sep 11 13:37:25.941290 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.941271    2573 scope.go:117] "RemoveContainer" containerID="fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"
Sep 11 13:37:25.941530 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:37:25.941513    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84\": container with ID starting with fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84 not found: ID does not exist" containerID="fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"
Sep 11 13:37:25.941577 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.941538    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84"} err="failed to get container status \"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84\": rpc error: code = NotFound desc = could not find container \"fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84\": container with ID starting with fa7b8670aa5925cbbcf129dbf4e53532f9c77e6af82fb94ed8458866104e8b84 not found: ID does not exist"
Sep 11 13:37:25.941577 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.941553    2573 scope.go:117] "RemoveContainer" containerID="01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"
Sep 11 13:37:25.941836 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:37:25.941818    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb\": container with ID starting with 01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb not found: ID does not exist" containerID="01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"
Sep 11 13:37:25.941907 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.941840    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb"} err="failed to get container status \"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb\": rpc error: code = NotFound desc = could not find container \"01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb\": container with ID starting with 01744df62d7a10342470d1138c4b9e0bc0173d6d270ee114d39bb9918cdf57bb not found: ID does not exist"
Sep 11 13:37:25.946518 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.946488    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:37:25.951959 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.951939    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-a-48243b-openshift-default-754f9dd96-z5wj8"]
Sep 11 13:37:25.971986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:25.971963    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" path="/var/lib/kubelet/pods/bb465727-6892-4c52-9dcd-c2a6b97f5ae7/volumes"
Sep 11 13:37:34.627747 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:34.627696    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:37:38.319125 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:38.319043    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:37:38.319526 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:38.319264    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy" containerID="cri-o://cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d" gracePeriod=30
Sep 11 13:37:43.564746 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.564722    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:37:43.607133 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607076    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607140    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607175    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bcwz7\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607236 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607222    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607250    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607277    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607312    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607537 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607401    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607537 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607429    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs\") pod \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\" (UID: \"d4c1284e-6fcf-49f3-9f2b-beca24fddb74\") "
Sep 11 13:37:43.607537 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607455    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:43.607737 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607696    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:43.607921 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607748    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data" (OuterVolumeSpecName: "istio-data") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:43.608038 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607771    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:43.608136 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.608032    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:37:43.608244 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.607725    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.609354 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.609329    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token" (OuterVolumeSpecName: "istio-token") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:43.609497 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.609447    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:37:43.609791 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.609767    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:37:43.609858 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.609768    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7" (OuterVolumeSpecName: "kube-api-access-bcwz7") pod "d4c1284e-6fcf-49f3-9f2b-beca24fddb74" (UID: "d4c1284e-6fcf-49f3-9f2b-beca24fddb74"). InnerVolumeSpecName "kube-api-access-bcwz7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:37:43.708753 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708727    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708753 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708748    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708753 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708756    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708765    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bcwz7\" (UniqueName: \"kubernetes.io/projected/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-kube-api-access-bcwz7\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708775    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708785    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708793    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:43.708923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:43.708801    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/d4c1284e-6fcf-49f3-9f2b-beca24fddb74-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:37:44.019872 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.019842    2573 generic.go:356] "Generic (PLEG): container finished" podID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerID="cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d" exitCode=0
Sep 11 13:37:44.020004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.019884    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"
Sep 11 13:37:44.020004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.019909    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerDied","Data":"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"}
Sep 11 13:37:44.020004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.019948    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7" event={"ID":"d4c1284e-6fcf-49f3-9f2b-beca24fddb74","Type":"ContainerDied","Data":"992b44ec51a3e59e44f176faa6cafb3f3633ddde749523b48db1b1ddff7bf269"}
Sep 11 13:37:44.020004 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.019970    2573 scope.go:117] "RemoveContainer" containerID="cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"
Sep 11 13:37:44.028610 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.028599    2573 scope.go:117] "RemoveContainer" containerID="cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"
Sep 11 13:37:44.036162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.036144    2573 scope.go:117] "RemoveContainer" containerID="cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"
Sep 11 13:37:44.036436 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:37:44.036418    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d\": container with ID starting with cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d not found: ID does not exist" containerID="cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"
Sep 11 13:37:44.036484 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.036443    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d"} err="failed to get container status \"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d\": rpc error: code = NotFound desc = could not find container \"cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d\": container with ID starting with cd54984dd241eacdb221b62259e39d7103612cca0859768b05d70cea3ca8889d not found: ID does not exist"
Sep 11 13:37:44.036484 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.036459    2573 scope.go:117] "RemoveContainer" containerID="cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"
Sep 11 13:37:44.036701 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:37:44.036683    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee\": container with ID starting with cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee not found: ID does not exist" containerID="cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"
Sep 11 13:37:44.036767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.036718    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee"} err="failed to get container status \"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee\": rpc error: code = NotFound desc = could not find container \"cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee\": container with ID starting with cc2d2010baa985b9d9f1e69875bc440c57e511f571cc6a87d25c3ec15c9831ee not found: ID does not exist"
Sep 11 13:37:44.143864 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.143840    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:37:44.149305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:44.149285    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-ipp-b-47b957-openshift-default-5c6f8df9c6-8hdz7"]
Sep 11 13:37:45.972836 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:45.972805    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" path="/var/lib/kubelet/pods/d4c1284e-6fcf-49f3-9f2b-beca24fddb74/volumes"
Sep 11 13:37:52.309147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:52.309114    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:37:56.847055 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:56.847025    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:37:56.847589 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:37:56.847218    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy" containerID="cri-o://7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e" gracePeriod=30
Sep 11 13:38:02.098775 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.098742    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:38:02.114671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.114601    2573 generic.go:356] "Generic (PLEG): container finished" podID="7da9e048-26b0-4346-8c42-25024e0c5439" containerID="7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e" exitCode=0
Sep 11 13:38:02.114671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.114641    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"
Sep 11 13:38:02.114838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.114683    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerDied","Data":"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"}
Sep 11 13:38:02.114838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.114755    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn" event={"ID":"7da9e048-26b0-4346-8c42-25024e0c5439","Type":"ContainerDied","Data":"b4d72df8d5d48376ddadbb9c08d356fd69284499cb0b168e463c3dc0d6e51e0e"}
Sep 11 13:38:02.114838 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.114781    2573 scope.go:117] "RemoveContainer" containerID="7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"
Sep 11 13:38:02.123188 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.123171    2573 scope.go:117] "RemoveContainer" containerID="0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"
Sep 11 13:38:02.130997 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.130981    2573 scope.go:117] "RemoveContainer" containerID="7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"
Sep 11 13:38:02.131254 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:38:02.131236    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e\": container with ID starting with 7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e not found: ID does not exist" containerID="7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"
Sep 11 13:38:02.131301 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.131263    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e"} err="failed to get container status \"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e\": rpc error: code = NotFound desc = could not find container \"7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e\": container with ID starting with 7c9aae723217a0fe12839a1e59d17a56fc7af69a46e11e261d671440c123519e not found: ID does not exist"
Sep 11 13:38:02.131301 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.131281    2573 scope.go:117] "RemoveContainer" containerID="0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"
Sep 11 13:38:02.131508 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:38:02.131495    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac\": container with ID starting with 0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac not found: ID does not exist" containerID="0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"
Sep 11 13:38:02.131549 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.131511    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac"} err="failed to get container status \"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac\": rpc error: code = NotFound desc = could not find container \"0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac\": container with ID starting with 0939fb71d99e9710733dc9aff9e6fa5d404ab1721e9875d15756197f2b11daac not found: ID does not exist"
Sep 11 13:38:02.153286 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153264    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153296    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153320    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-227d9\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153341    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153422 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153364    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153491    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153602 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153577    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket" (OuterVolumeSpecName: "credential-socket") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:02.153697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153630    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153697 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153659    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153825 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153698    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs\") pod \"7da9e048-26b0-4346-8c42-25024e0c5439\" (UID: \"7da9e048-26b0-4346-8c42-25024e0c5439\") "
Sep 11 13:38:02.153825 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153774    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket" (OuterVolumeSpecName: "workload-socket") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:02.154006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153959    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data" (OuterVolumeSpecName: "istio-data") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:02.154006 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.153969    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs" (OuterVolumeSpecName: "workload-certs") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:02.154131 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.154100    2573 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-data\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.154131 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.154120    2573 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.154239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.154134    2573 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-credential-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.154239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.154147    2573 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-workload-socket\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.154239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.154170    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert" (OuterVolumeSpecName: "istiod-ca-cert") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:38:02.155802 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.155765    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy" (OuterVolumeSpecName: "istio-envoy") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:38:02.156237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.156216    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9" (OuterVolumeSpecName: "kube-api-access-227d9") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "kube-api-access-227d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:02.156237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.156225    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token" (OuterVolumeSpecName: "istio-token") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:02.156383 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.156345    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo" (OuterVolumeSpecName: "istio-podinfo") pod "7da9e048-26b0-4346-8c42-25024e0c5439" (UID: "7da9e048-26b0-4346-8c42-25024e0c5439"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue ""
Sep 11 13:38:02.255540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.255511    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-227d9\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-kube-api-access-227d9\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.255540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.255538    2573 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7da9e048-26b0-4346-8c42-25024e0c5439-istio-podinfo\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.255540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.255547    2573 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7da9e048-26b0-4346-8c42-25024e0c5439-istiod-ca-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.255783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.255555    2573 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7da9e048-26b0-4346-8c42-25024e0c5439-istio-token\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.255783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.255563    2573 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7da9e048-26b0-4346-8c42-25024e0c5439-istio-envoy\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:02.442612 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.442587    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:38:02.453687 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:02.453656    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-shared-a-w3-6ffb6a-openshift-default-6d84c544b6-xbgxn"]
Sep 11 13:38:03.971909 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:03.971874    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" path="/var/lib/kubelet/pods/7da9e048-26b0-4346-8c42-25024e0c5439/volumes"
Sep 11 13:38:11.212666 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:11.212627    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:38:19.985895 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:19.985846    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:38:19.986299 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:19.986082    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" podUID="7e62a940-6685-46a0-a12d-25ba95688080" containerName="manager" containerID="cri-o://39b0c2bf7507f357d8e47bb847f3f794a876686fc0af7f4c3e1a5b7b0d4c78b5" gracePeriod=30
Sep 11 13:38:20.205928 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.205897    2573 generic.go:356] "Generic (PLEG): container finished" podID="7e62a940-6685-46a0-a12d-25ba95688080" containerID="39b0c2bf7507f357d8e47bb847f3f794a876686fc0af7f4c3e1a5b7b0d4c78b5" exitCode=0
Sep 11 13:38:20.206088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.205975    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" event={"ID":"7e62a940-6685-46a0-a12d-25ba95688080","Type":"ContainerDied","Data":"39b0c2bf7507f357d8e47bb847f3f794a876686fc0af7f4c3e1a5b7b0d4c78b5"}
Sep 11 13:38:20.232443 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.232423    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:38:20.288796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.288771    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs\") pod \"7e62a940-6685-46a0-a12d-25ba95688080\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") "
Sep 11 13:38:20.288961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.288811    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t6z4l\" (UniqueName: \"kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l\") pod \"7e62a940-6685-46a0-a12d-25ba95688080\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") "
Sep 11 13:38:20.288961 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.288861    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert\") pod \"7e62a940-6685-46a0-a12d-25ba95688080\" (UID: \"7e62a940-6685-46a0-a12d-25ba95688080\") "
Sep 11 13:38:20.290916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.290875    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "7e62a940-6685-46a0-a12d-25ba95688080" (UID: "7e62a940-6685-46a0-a12d-25ba95688080"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:38:20.290916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.290904    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l" (OuterVolumeSpecName: "kube-api-access-t6z4l") pod "7e62a940-6685-46a0-a12d-25ba95688080" (UID: "7e62a940-6685-46a0-a12d-25ba95688080"). InnerVolumeSpecName "kube-api-access-t6z4l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:38:20.291051 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.290950    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "7e62a940-6685-46a0-a12d-25ba95688080" (UID: "7e62a940-6685-46a0-a12d-25ba95688080"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:38:20.389611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.389586    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:20.389611 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.389609    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t6z4l\" (UniqueName: \"kubernetes.io/projected/7e62a940-6685-46a0-a12d-25ba95688080-kube-api-access-t6z4l\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:20.389818 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:20.389618    2573 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/7e62a940-6685-46a0-a12d-25ba95688080-webhook-cert\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:38:21.215727 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.215684    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:38:21.218335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.218305    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf" event={"ID":"7e62a940-6685-46a0-a12d-25ba95688080","Type":"ContainerDied","Data":"8a854c85fa20bd6398dc8f4993007d41a05df5a6874e67ad13b1589bf1deb89a"}
Sep 11 13:38:21.218335 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.218326    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7fcc9dff55-7pzzf"
Sep 11 13:38:21.218558 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.218364    2573 scope.go:117] "RemoveContainer" containerID="39b0c2bf7507f357d8e47bb847f3f794a876686fc0af7f4c3e1a5b7b0d4c78b5"
Sep 11 13:38:21.240266 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.240246    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:38:21.243923 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.243899    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7fcc9dff55-7pzzf"]
Sep 11 13:38:21.973671 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:38:21.973636    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7e62a940-6685-46a0-a12d-25ba95688080" path="/var/lib/kubelet/pods/7e62a940-6685-46a0-a12d-25ba95688080/volumes"
Sep 11 13:39:46.409625 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:39:46.409586    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:10.507996 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:10.507967    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:18.609252 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:18.609218    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:26.013410 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:26.013376    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:34.001723 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:34.001681    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:46.992603 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:46.992524    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:40:55.194156 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:40:55.194121    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:42:24.314109 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:24.314026    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:42:30.916990 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:30.916953    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:42:40.116460 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:40.116423    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:42:50.578423 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578376    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:42:50.578888 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578873    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578890    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578899    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" containerName="maas-api"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578904    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" containerName="maas-api"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578912    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" containerName="main"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578918    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" containerName="main"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578928    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578933    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578940    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:42:50.578944 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578945    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578955    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" containerName="payload-processing"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578961    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" containerName="payload-processing"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578968    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578973    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578979    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="588675c9-4c53-4297-902f-8743e0a2a17b" containerName="payload-processing"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578984    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="588675c9-4c53-4297-902f-8743e0a2a17b" containerName="payload-processing"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578993    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7e62a940-6685-46a0-a12d-25ba95688080" containerName="manager"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.578997    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7e62a940-6685-46a0-a12d-25ba95688080" containerName="manager"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579053    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7e62a940-6685-46a0-a12d-25ba95688080" containerName="manager"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579061    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579066    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579071    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="75a224d4-595f-4ae2-8a02-35d6a9d67adf" containerName="main"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579077    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579082    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="2cb598fe-1de5-4f74-8bf0-eef0eb463be4" containerName="maas-api"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579087    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579092    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="8b7b0c53-9fce-49a5-be5a-69c35c2bb95b" containerName="payload-processing"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579096    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:42:50.579215 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.579101    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="588675c9-4c53-4297-902f-8743e0a2a17b" containerName="payload-processing"
Sep 11 13:42:50.582228 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.582211    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.584504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.584472    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-same-id-b-eaefc095-kserve-self-signed-certs\""
Sep 11 13:42:50.717554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717520    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.717554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717558    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p2jlg\" (UniqueName: \"kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.717780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717631    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.717780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717734    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.717780 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717760    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.717880 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.717783    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818361 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818333    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818365    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p2jlg\" (UniqueName: \"kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818394    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818423    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818442    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818504 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818462    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818835 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818817    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818898 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818879    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.818940 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.818924    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.820749 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.820719    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:50.820994 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:50.820979    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:51.083633 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:51.083603    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:42:51.089701 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:51.089671    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p2jlg\" (UniqueName: \"kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg\") pod \"e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") " pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:51.192462 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:51.192427    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:42:51.475225 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:51.475198    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:42:51.476425 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:42:51.476396    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd6ad8de7_4aad_4044_bab2_bdccedba7ee8.slice/crio-972e9d356a01be2def1bd9c9715d6bcf5759a237e7b4c58eeea51e7c01ce0564 WatchSource:0}: Error finding container 972e9d356a01be2def1bd9c9715d6bcf5759a237e7b4c58eeea51e7c01ce0564: Status 404 returned error can't find the container with id 972e9d356a01be2def1bd9c9715d6bcf5759a237e7b4c58eeea51e7c01ce0564
Sep 11 13:42:51.506088 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:51.506053    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" event={"ID":"d6ad8de7-4aad-4044-bab2-bdccedba7ee8","Type":"ContainerStarted","Data":"972e9d356a01be2def1bd9c9715d6bcf5759a237e7b4c58eeea51e7c01ce0564"}
Sep 11 13:42:52.512327 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:52.512290    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" event={"ID":"d6ad8de7-4aad-4044-bab2-bdccedba7ee8","Type":"ContainerStarted","Data":"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"}
Sep 11 13:42:52.535555 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:52.535512    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" podStartSLOduration=3.535498426 podStartE2EDuration="3.535498426s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:42:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:42:52.534731707 +0000 UTC m=+2287.054769052" watchObservedRunningTime="2026-09-11 13:42:52.535498426 +0000 UTC m=+2287.055535771"
Sep 11 13:42:54.324451 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:42:54.324412    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:01.193348 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:01.193308    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:01.193783 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:01.193383    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:01.205983 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:01.205959    2573 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:01.567170 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:01.567141    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:10.004219 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:10.004188    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:43:10.504360 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:10.504328    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:19.536425 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:19.536393    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:25.023026 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.022970    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" podUID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" containerName="main" containerID="cri-o://6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c" gracePeriod=45
Sep 11 13:43:25.259491 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.259467    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:25.396101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396029    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396101 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396079    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p2jlg\" (UniqueName: \"kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396113    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396150    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396176    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396293 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396255    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache\") pod \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\" (UID: \"d6ad8de7-4aad-4044-bab2-bdccedba7ee8\") "
Sep 11 13:43:25.396519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396301    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:25.396519 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396405    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home" (OuterVolumeSpecName: "home") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:25.396620 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396563    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache" (OuterVolumeSpecName: "model-cache") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:25.396675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396637    2573 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tmp-dir\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.396675 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.396651    2573 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-home\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.398305 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.398284    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm" (OuterVolumeSpecName: "dshm") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
Sep 11 13:43:25.398404 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.398327    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:25.398444 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.398399    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg" (OuterVolumeSpecName: "kube-api-access-p2jlg") pod "d6ad8de7-4aad-4044-bab2-bdccedba7ee8" (UID: "d6ad8de7-4aad-4044-bab2-bdccedba7ee8"). InnerVolumeSpecName "kube-api-access-p2jlg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:25.497640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.497613    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p2jlg\" (UniqueName: \"kubernetes.io/projected/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-kube-api-access-p2jlg\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.497640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.497637    2573 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-tls-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.497640 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.497646    2573 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-dshm\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.497850 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.497655    2573 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d6ad8de7-4aad-4044-bab2-bdccedba7ee8-model-cache\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:25.675732 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.675644    2573 generic.go:356] "Generic (PLEG): container finished" podID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" containerID="6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c" exitCode=0
Sep 11 13:43:25.675732 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.675690    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"
Sep 11 13:43:25.675924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.675685    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" event={"ID":"d6ad8de7-4aad-4044-bab2-bdccedba7ee8","Type":"ContainerDied","Data":"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"}
Sep 11 13:43:25.675924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.675828    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs" event={"ID":"d6ad8de7-4aad-4044-bab2-bdccedba7ee8","Type":"ContainerDied","Data":"972e9d356a01be2def1bd9c9715d6bcf5759a237e7b4c58eeea51e7c01ce0564"}
Sep 11 13:43:25.675924 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.675860    2573 scope.go:117] "RemoveContainer" containerID="6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"
Sep 11 13:43:25.684638 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.684621    2573 scope.go:117] "RemoveContainer" containerID="6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"
Sep 11 13:43:25.684922 ip-10-0-142-204 kubenswrapper[2573]: E0911 13:43:25.684906    2573 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c\": container with ID starting with 6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c not found: ID does not exist" containerID="6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"
Sep 11 13:43:25.684978 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.684929    2573 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c"} err="failed to get container status \"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c\": rpc error: code = NotFound desc = could not find container \"6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c\": container with ID starting with 6235c51ba761ce5cdc828d20d3491700d7815c8c2aca0da6cd84d88db94b4d5c not found: ID does not exist"
Sep 11 13:43:25.700989 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.700967    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:43:25.706901 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.706881    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["llm/e2e-same-id-b-eaefc095-kserve-86c5c964b7-lckqs"]
Sep 11 13:43:25.971741 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:25.971664    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" path="/var/lib/kubelet/pods/d6ad8de7-4aad-4044-bab2-bdccedba7ee8/volumes"
Sep 11 13:43:28.712492 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:28.712437    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:29.240237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240203    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"]
Sep 11 13:43:29.240540 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240528    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:43:29.240598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240542    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="bb465727-6892-4c52-9dcd-c2a6b97f5ae7" containerName="istio-proxy"
Sep 11 13:43:29.240598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240561    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:43:29.240598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240567    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="7da9e048-26b0-4346-8c42-25024e0c5439" containerName="istio-proxy"
Sep 11 13:43:29.240598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240574    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" containerName="main"
Sep 11 13:43:29.240598 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240580    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" containerName="main"
Sep 11 13:43:29.240772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240629    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d4c1284e-6fcf-49f3-9f2b-beca24fddb74" containerName="istio-proxy"
Sep 11 13:43:29.240772 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.240641    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="d6ad8de7-4aad-4044-bab2-bdccedba7ee8" containerName="main"
Sep 11 13:43:29.244689 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.244664    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.257111 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.257089    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"]
Sep 11 13:43:29.430449 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.430417    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-maas-api-tls\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.430627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.430497    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0549166-5ac7-45bf-878e-edd226785627-openshift-service-ca\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.430627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.430562    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-metrics-certs\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.430627 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.430616    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cpvxp\" (UniqueName: \"kubernetes.io/projected/d0549166-5ac7-45bf-878e-edd226785627-kube-api-access-cpvxp\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.531393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.531322    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-metrics-certs\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.531393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.531358    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cpvxp\" (UniqueName: \"kubernetes.io/projected/d0549166-5ac7-45bf-878e-edd226785627-kube-api-access-cpvxp\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.531393 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.531392    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-maas-api-tls\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.531580 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.531427    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0549166-5ac7-45bf-878e-edd226785627-openshift-service-ca\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.532162 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.532139    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/d0549166-5ac7-45bf-878e-edd226785627-openshift-service-ca\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.533897 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.533876    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-maas-api-tls\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.534000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.533916    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/d0549166-5ac7-45bf-878e-edd226785627-metrics-certs\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.541764 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.541743    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cpvxp\" (UniqueName: \"kubernetes.io/projected/d0549166-5ac7-45bf-878e-edd226785627-kube-api-access-cpvxp\") pod \"maas-api-6876b74b58-5jwnd\" (UID: \"d0549166-5ac7-45bf-878e-edd226785627\") " pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.554547 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.554524    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:29.831927 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.831901    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"]
Sep 11 13:43:29.834500 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:43:29.834469    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0549166_5ac7_45bf_878e_edd226785627.slice/crio-b1bc8d60873863085297dc73b4366f69553db217be14a81aa153beca785919aa WatchSource:0}: Error finding container b1bc8d60873863085297dc73b4366f69553db217be14a81aa153beca785919aa: Status 404 returned error can't find the container with id b1bc8d60873863085297dc73b4366f69553db217be14a81aa153beca785919aa
Sep 11 13:43:29.835701 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:29.835683    2573 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
Sep 11 13:43:30.702433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:30.702356    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd" event={"ID":"d0549166-5ac7-45bf-878e-edd226785627","Type":"ContainerStarted","Data":"65625741f7ff00b5bc99daf3b9da81f2a0803f64e168b0d3ddace35e1f3027ac"}
Sep 11 13:43:30.702433 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:30.702410    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd" event={"ID":"d0549166-5ac7-45bf-878e-edd226785627","Type":"ContainerStarted","Data":"b1bc8d60873863085297dc73b4366f69553db217be14a81aa153beca785919aa"}
Sep 11 13:43:30.702609 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:30.702440    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:30.724633 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:30.724586    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd" podStartSLOduration=1.245167724 podStartE2EDuration="1.724571369s" totalImagesPullingTime="479.403645ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:43:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:43:30.723789936 +0000 UTC m=+2325.243827283" watchObservedRunningTime="2026-09-11 13:43:30.724571369 +0000 UTC m=+2325.244608715"
Sep 11 13:43:36.712237 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:36.712205    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-6876b74b58-5jwnd"
Sep 11 13:43:36.935631 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:36.935592    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:43.606344 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:43.606271    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:44.866205 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:44.866174    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:43:44.866592 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:44.866353    2573 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" podUID="b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" containerName="maas-api" containerID="cri-o://f88b447bc690cb17a5fbec399a969cc15262dd44919194ed526c12eb4425197a" gracePeriod=30
Sep 11 13:43:46.104634 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.104608    2573 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:43:46.146740 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.146713    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") pod \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") "
Sep 11 13:43:46.146887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.146799    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gtblf\" (UniqueName: \"kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf\") pod \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") "
Sep 11 13:43:46.146887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.146826    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca\") pod \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") "
Sep 11 13:43:46.146887 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.146863    2573 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls\") pod \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\" (UID: \"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35\") "
Sep 11 13:43:46.147231 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.147205    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" (UID: "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
Sep 11 13:43:46.148769 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.148743    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs" (OuterVolumeSpecName: "metrics-certs") pod "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" (UID: "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:46.148888 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.148868    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf" (OuterVolumeSpecName: "kube-api-access-gtblf") pod "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" (UID: "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35"). InnerVolumeSpecName "kube-api-access-gtblf". PluginName "kubernetes.io/projected", VolumeGIDValue ""
Sep 11 13:43:46.149064 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.149028    2573 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" (UID: "b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
Sep 11 13:43:46.247411 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.247389    2573 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gtblf\" (UniqueName: \"kubernetes.io/projected/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-kube-api-access-gtblf\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:46.247411 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.247411    2573 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-openshift-service-ca\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:46.247581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.247420    2573 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-maas-api-tls\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:46.247581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.247429    2573 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35-metrics-certs\") on node \"ip-10-0-142-204.ec2.internal\" DevicePath \"\""
Sep 11 13:43:46.325049 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.325023    2573 scope.go:117] "RemoveContainer" containerID="f88b447bc690cb17a5fbec399a969cc15262dd44919194ed526c12eb4425197a"
Sep 11 13:43:46.782110 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.782086    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"
Sep 11 13:43:46.782239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.782087    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" event={"ID":"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35","Type":"ContainerDied","Data":"f88b447bc690cb17a5fbec399a969cc15262dd44919194ed526c12eb4425197a"}
Sep 11 13:43:46.782239 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.782208    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g" event={"ID":"b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35","Type":"ContainerDied","Data":"c3218b0984ac3affa1aedf18925976bcc44a0630ee1964f42e6d98ea2f3aafe7"}
Sep 11 13:43:46.816184 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.816159    2573 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:43:46.821147 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:46.821127    2573 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-549f4476fd-x4v6g"]
Sep 11 13:43:47.971185 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:47.971153    2573 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" path="/var/lib/kubelet/pods/b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35/volumes"
Sep 11 13:43:49.605235 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:49.605202    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:56.805300 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:56.805268    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-84994f7ffc-4lxzk"]
Sep 11 13:43:59.098032 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:43:59.097994    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bdff694-zq5s9_167057a1-2653-4dbb-be89-7696c3220d48/authorino/0.log"
Sep 11 13:44:05.092796 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:05.092753    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2d5cb_d5ad7f4d-ff8d-4e35-891b-771687dabb8e/manager/0.log"
Sep 11 13:44:05.478760 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:05.478675    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-szq2v_e80e8554-2a02-4f76-b8da-5f1f9b9f69ff/copy-manifests/0.log"
Sep 11 13:44:05.487597 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:05.487580    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-szq2v_e80e8554-2a02-4f76-b8da-5f1f9b9f69ff/manager/0.log"
Sep 11 13:44:05.844018 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:05.843989    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-6876b74b58-5jwnd_d0549166-5ac7-45bf-878e-edd226785627/maas-api/0.log"
Sep 11 13:44:06.187014 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:06.186950    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_postgres-69fb4594cf-nhrlh_70387693-2818-4205-9e7e-405b1e82214e/postgres/0.log"
Sep 11 13:44:06.976376 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:06.976347    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bdff694-zq5s9_167057a1-2653-4dbb-be89-7696c3220d48/authorino/0.log"
Sep 11 13:44:07.098848 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:07.098824    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vsqmw_ed7d162a-cc13-4b84-8c33-7f44f6fefb03/manager/0.log"
Sep 11 13:44:07.211800 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:07.211777    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5vb4s_20aa07aa-5187-4777-b761-d7b3287a450f/manager/0.log"
Sep 11 13:44:07.323446 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:07.323426    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9d429_d62b639f-8df9-4b08-85ec-776ed90937f9/kuadrant-console-plugin/0.log"
Sep 11 13:44:07.571238 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:07.571217    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-4lxzk_a262c75f-c0f1-4cef-a93a-67091b4559ba/limitador/0.log"
Sep 11 13:44:07.691494 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:07.691440    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-9p4mn_146648ff-6004-4daf-b36f-697f16fd35c8/manager/0.log"
Sep 11 13:44:08.417854 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:08.417824    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-xd42v_875c6116-c68a-4aaa-9675-0509c02f5413/kube-auth-proxy/0.log"
Sep 11 13:44:08.537994 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:08.537973    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/1.log"
Sep 11 13:44:08.781869 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:08.781845    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:44:09.501682 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:09.501653    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-embedding-simulated-kserve-bcc9ff8d8-pg2gx_88c50909-2cc7-473f-a191-b2acf96c01f3/main/0.log"
Sep 11 13:44:09.863394 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:09.863317    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-8nvjq_639bf75f-2532-4560-8a3c-5e9ec1bad159/main/0.log"
Sep 11 13:44:09.976525 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:09.976493    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-57fc66f697-8w425_b713e5e7-9257-408c-844e-946d74e00463/main/0.log"
Sep 11 13:44:15.507048 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.507012    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5f924/must-gather-jt9lg"]
Sep 11 13:44:15.507394 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.507322    2573 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" containerName="maas-api"
Sep 11 13:44:15.507394 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.507333    2573 state_mem.go:107] "Deleted CPUSet assignment" podUID="b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" containerName="maas-api"
Sep 11 13:44:15.507394 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.507392    2573 memory_manager.go:368] "RemoveStaleState removing state" podUID="b67d81c9-44ed-42c0-bd4e-f9fed7fdaf35" containerName="maas-api"
Sep 11 13:44:15.510632 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.510611    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.514683 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.514661    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"kube-root-ca.crt\""
Sep 11 13:44:15.514815 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.514693    2573 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-5f924\"/\"default-dockercfg-cxjdm\""
Sep 11 13:44:15.514815 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.514697    2573 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-5f924\"/\"openshift-service-ca.crt\""
Sep 11 13:44:15.523192 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.523170    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/must-gather-jt9lg"]
Sep 11 13:44:15.571061 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.571035    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16862d66-68c0-4305-815d-2aa6e4d29595-must-gather-output\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.571156 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.571068    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4sfqb\" (UniqueName: \"kubernetes.io/projected/16862d66-68c0-4305-815d-2aa6e4d29595-kube-api-access-4sfqb\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.672338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.672316    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16862d66-68c0-4305-815d-2aa6e4d29595-must-gather-output\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.672424 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.672343    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4sfqb\" (UniqueName: \"kubernetes.io/projected/16862d66-68c0-4305-815d-2aa6e4d29595-kube-api-access-4sfqb\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.672644 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.672625    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/16862d66-68c0-4305-815d-2aa6e4d29595-must-gather-output\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.687380 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.687360    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4sfqb\" (UniqueName: \"kubernetes.io/projected/16862d66-68c0-4305-815d-2aa6e4d29595-kube-api-access-4sfqb\") pod \"must-gather-jt9lg\" (UID: \"16862d66-68c0-4305-815d-2aa6e4d29595\") " pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:15.820049 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:15.819992    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/must-gather-jt9lg"
Sep 11 13:44:16.102835 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:16.102807    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/must-gather-jt9lg"]
Sep 11 13:44:16.103781 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:44:16.103754    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod16862d66_68c0_4305_815d_2aa6e4d29595.slice/crio-a02bfc4871e9f6261a6d27c3d2045ee1035537576beafc97eb0eaf716e044914 WatchSource:0}: Error finding container a02bfc4871e9f6261a6d27c3d2045ee1035537576beafc97eb0eaf716e044914: Status 404 returned error can't find the container with id a02bfc4871e9f6261a6d27c3d2045ee1035537576beafc97eb0eaf716e044914
Sep 11 13:44:16.925968 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:16.925932    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/must-gather-jt9lg" event={"ID":"16862d66-68c0-4305-815d-2aa6e4d29595","Type":"ContainerStarted","Data":"f98c6630ed535ed4ad4fbbf8fd6e5066100be73b06e018b2cebf730d643ac75b"}
Sep 11 13:44:16.926285 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:16.925992    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/must-gather-jt9lg" event={"ID":"16862d66-68c0-4305-815d-2aa6e4d29595","Type":"ContainerStarted","Data":"a02bfc4871e9f6261a6d27c3d2045ee1035537576beafc97eb0eaf716e044914"}
Sep 11 13:44:17.934742 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:17.933916    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/must-gather-jt9lg" event={"ID":"16862d66-68c0-4305-815d-2aa6e4d29595","Type":"ContainerStarted","Data":"ad7d6710e46b318de2f67eea0acf9ed888b84dd9fa5b7d6b7eeb88e595376505"}
Sep 11 13:44:17.959406 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:17.959356    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5f924/must-gather-jt9lg" podStartSLOduration=2.311841597 podStartE2EDuration="2.959340284s" totalImagesPullingTime="647.498687ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-09-11 13:44:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:17.957021752 +0000 UTC m=+2372.477059095" watchObservedRunningTime="2026-09-11 13:44:17.959340284 +0000 UTC m=+2372.479377627"
Sep 11 13:44:18.627146 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:18.627111    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-dv8kv_b50d37b9-a041-4e48-97f6-65d919e4777d/global-pull-secret-syncer/0.log"
Sep 11 13:44:18.857346 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:18.857314    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j7r7w_485cb65a-3796-410d-a65a-19dc773993d2/konnectivity-agent/0.log"
Sep 11 13:44:18.912103 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:18.912015    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-142-204.ec2.internal_af658a1e06ce3bdd72fe16298188a4e8/haproxy/0.log"
Sep 11 13:44:23.733739 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:23.733546    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-6bdff694-zq5s9_167057a1-2653-4dbb-be89-7696c3220d48/authorino/0.log"
Sep 11 13:44:23.780115 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:23.780068    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-76b478bd78-vsqmw_ed7d162a-cc13-4b84-8c33-7f44f6fefb03/manager/0.log"
Sep 11 13:44:23.820539 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:23.820479    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-7b84cd7c9c-5vb4s_20aa07aa-5187-4777-b761-d7b3287a450f/manager/0.log"
Sep 11 13:44:23.855668 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:23.855633    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-fbdd4f445-9d429_d62b639f-8df9-4b08-85ec-776ed90937f9/kuadrant-console-plugin/0.log"
Sep 11 13:44:23.951511 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:23.951478    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-84994f7ffc-4lxzk_a262c75f-c0f1-4cef-a93a-67091b4559ba/limitador/0.log"
Sep 11 13:44:24.024213 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:24.024107    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-74495c69f8-9p4mn_146648ff-6004-4daf-b36f-697f16fd35c8/manager/0.log"
Sep 11 13:44:26.834618 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:26.834588    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksck6_a5e50d9f-ce30-4137-9af0-1982af45e24e/node-exporter/0.log"
Sep 11 13:44:26.933475 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:26.933438    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksck6_a5e50d9f-ce30-4137-9af0-1982af45e24e/kube-rbac-proxy/0.log"
Sep 11 13:44:26.974662 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:26.974629    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-ksck6_a5e50d9f-ce30-4137-9af0-1982af45e24e/init-textfile/0.log"
Sep 11 13:44:28.185322 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.185259    2573 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"]
Sep 11 13:44:28.192041 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.191999    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.207093 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.207067    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"]
Sep 11 13:44:28.292182 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.292146    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-podres\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.292484 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.292466    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-proc\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.292654 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.292641    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-sys\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.292807 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.292790    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdlj6\" (UniqueName: \"kubernetes.io/projected/e8ce39a2-0f89-4e17-a082-ff88ca07a610-kube-api-access-fdlj6\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.293000 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.292984    2573 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-lib-modules\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394185 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394136    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-sys\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394207    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdlj6\" (UniqueName: \"kubernetes.io/projected/e8ce39a2-0f89-4e17-a082-ff88ca07a610-kube-api-access-fdlj6\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394269    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-lib-modules\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394370 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394329    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-podres\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394385    2573 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-proc\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394538 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394485    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-proc\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.394645 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394549    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-sys\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.395011 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.394984    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-lib-modules\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.395107 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.395088    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e8ce39a2-0f89-4e17-a082-ff88ca07a610-podres\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.407806 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.407775    2573 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdlj6\" (UniqueName: \"kubernetes.io/projected/e8ce39a2-0f89-4e17-a082-ff88ca07a610-kube-api-access-fdlj6\") pod \"perf-node-gather-daemonset-xnbdn\" (UID: \"e8ce39a2-0f89-4e17-a082-ff88ca07a610\") " pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.503342 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.503295    2573 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:28.792823 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:28.792792    2573 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"]
Sep 11 13:44:28.793693 ip-10-0-142-204 kubenswrapper[2573]: W0911 13:44:28.793665    2573 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode8ce39a2_0f89_4e17_a082_ff88ca07a610.slice/crio-21ffef897be4e62203b6e9b09b5c0b6cdc9e4ccc59636ddb4755cb283838df9a WatchSource:0}: Error finding container 21ffef897be4e62203b6e9b09b5c0b6cdc9e4ccc59636ddb4755cb283838df9a: Status 404 returned error can't find the container with id 21ffef897be4e62203b6e9b09b5c0b6cdc9e4ccc59636ddb4755cb283838df9a
Sep 11 13:44:29.024609 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:29.024562    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn" event={"ID":"e8ce39a2-0f89-4e17-a082-ff88ca07a610","Type":"ContainerStarted","Data":"b370b1084849e55ee33ba75ef119ddb55eab7153019411cf4935f078e81b9cc6"}
Sep 11 13:44:29.024814 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:29.024636    2573 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn" event={"ID":"e8ce39a2-0f89-4e17-a082-ff88ca07a610","Type":"ContainerStarted","Data":"21ffef897be4e62203b6e9b09b5c0b6cdc9e4ccc59636ddb4755cb283838df9a"}
Sep 11 13:44:29.025429 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:29.025399    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:29.054554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:29.054458    2573 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn" podStartSLOduration=1.054443206 podStartE2EDuration="1.054443206s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-09-11 13:44:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-09-11 13:44:29.050175474 +0000 UTC m=+2383.570212818" watchObservedRunningTime="2026-09-11 13:44:29.054443206 +0000 UTC m=+2383.574480550"
Sep 11 13:44:32.501767 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:32.501739    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4n6v_2e28c135-924b-42c3-8197-930e3e65cd7e/dns/0.log"
Sep 11 13:44:32.534192 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:32.534149    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-m4n6v_2e28c135-924b-42c3-8197-930e3e65cd7e/kube-rbac-proxy/0.log"
Sep 11 13:44:32.684355 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:32.684325    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-zh8nr_6bcf9b29-dcc6-4882-bb12-2066011b34af/dns-node-resolver/0.log"
Sep 11 13:44:33.298814 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:33.298786    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-nvkr7_d664ec04-e3f3-46ae-a842-adb7cb6d7739/node-ca/0.log"
Sep 11 13:44:35.070338 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:35.070304    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-xd42v_875c6116-c68a-4aaa-9675-0509c02f5413/kube-auth-proxy/0.log"
Sep 11 13:44:35.155581 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:35.155548    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/0.log"
Sep 11 13:44:35.164916 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:35.164887    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/1.log"
Sep 11 13:44:35.355986 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:35.355921    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:44:36.045619 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:36.045588    2573 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-5f924/perf-node-gather-daemonset-xnbdn"
Sep 11 13:44:36.830477 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:36.830448    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-frj5r_9e61091c-384a-40bd-a73f-7857eb641db4/serve-healthcheck-canary/0.log"
Sep 11 13:44:38.231470 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.231345    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-87mdc_4e78d822-55ab-4ad7-906f-eb603b36b48a/kube-rbac-proxy/0.log"
Sep 11 13:44:38.298554 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.298530    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-87mdc_4e78d822-55ab-4ad7-906f-eb603b36b48a/exporter/0.log"
Sep 11 13:44:38.354381 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.354351    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-87mdc_4e78d822-55ab-4ad7-906f-eb603b36b48a/extractor/0.log"
Sep 11 13:44:38.813881 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.813846    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-xd42v_875c6116-c68a-4aaa-9675-0509c02f5413/kube-auth-proxy/0.log"
Sep 11 13:44:38.843571 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.843483    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/0.log"
Sep 11 13:44:38.851943 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.851915    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/1.log"
Sep 11 13:44:38.924418 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:38.924392    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:44:40.728920 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:40.728806    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-74db4b4cbd-xd42v_875c6116-c68a-4aaa-9675-0509c02f5413/kube-auth-proxy/0.log"
Sep 11 13:44:40.758720 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:40.758660    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/0.log"
Sep 11 13:44:40.777428 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:40.774406    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-78bb78ff78-zbkh4_1a837d58-46f0-4315-97bb-52317f2e9547/istio-proxy/1.log"
Sep 11 13:44:40.845347 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:40.845316    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-5b54b7fbb7-plp2d_9ad61c6c-9b85-4cbe-bace-6d34a6a54eb4/payload-processing/0.log"
Sep 11 13:44:41.865339 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:41.865306    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-7998c67dc4-2d5cb_d5ad7f4d-ff8d-4e35-891b-771687dabb8e/manager/0.log"
Sep 11 13:44:42.208171 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:42.208038    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-szq2v_e80e8554-2a02-4f76-b8da-5f1f9b9f69ff/manager/0.log"
Sep 11 13:44:42.239812 ip-10-0-142-204 kubenswrapper[2573]: I0911 13:44:42.239779    2573 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-5c4b86887d-szq2v_e80e8554-2a02-4f76-b8da-5f1f9b9f69ff/copy-manifests/0.log"